Roughly chop the orange rinds after juicing and place in the cleaned duck cavity. Place the stuffed duck on a baking rack over a baking sheet with 1/2-inch of water. Bake until skin turns golden brown and lightly crisps, about 30 minutes. Reduce temperature to 300 degrees and continue cooking until duck reaches an internal temperature of 170 degrees, about 1 hour.
In a medium heavy saucepan combine the orange juice, zest and sugar over medium high heat and reduce nearly 3/4 in volume, to about 3/4 cup. Add bitters to orange juice gastrique, and set aside. Place duck stock in clean saucepan and bring to a boil over high heat. Add hot stock to reduced orange gastrique, and continue to simmer over medium low heat for 10 minutes to make sauce.
Remove duck from roasting pan and discard drippings in bottom of pan. Return duck to roasting pan and place pan over 2 burners over medium high heat. Add orange liqueur to pan and cook off the alcohol, scraping the pan continuously with a large wooden spoon. Add 1 cup of the orange sauce to the roasting pan and cook 1 minute. Remove duck from the pan and discard orange rinds in cavity. Place duck on serving platter and let sit 10 minutes before carving. Combine roasting pan juices and orange sauce in a gravy boat and serve with carved duck.