In an 8-quart soup pot, heat the oil over medium heat. Add the celery, carrots, garlic and shallots, and saute until just tender, 5 to 8 minutes. Deglaze with the sherry, and then add in the stock, bay leaves and thyme. Add in the chicken, bring to a simmer and cook until the chicken is cooked through, 15 to 20 minutes. Remove the chicken and let it cool. Simmer the soup for 10 minutes more. Season, and remove the thyme and bay leaves.
Shred the cooled chicken. Add any accumulated juices from the chicken plate back to the pot along with the lemon zest and juice. Add in the butter and adjust seasoning.
Portion the orzo and shredded chicken into 4 bowls. Ladle the soup over and garnish with the tarragon and a lemon wedge. Serve with a ton of butter crackers.