For the cookies: Line 2 baking sheets with parchment paper.
In the bowl of an electric mixer fitted with a paddle attachment, cream together the granulated sugar, butter and orange zest until light and fluffy, about 2 minutes. Add the eggs one at a time, mixing fully between each. In a separate bowl, mix together the flour, baking powder and salt. Add to the mixer bowl and blend until just combined.
Using floured hands, roll the dough into 1-inch balls and divide between the prepared baking sheets, setting the cookies 1 inch apart. Refrigerate for 1 hour to chill before baking so that the cookies retain their shape.
Preheat the oven to 350 degrees F.
Bake until the cookies are just barely turning brown, 10 to 12 minutes; don't let them overbrown. Place on a wire rack until completely cooled, about 15 minutes.
For the icing: Whisk together the confectioners' sugar, milk and lemon zest in a bowl until smooth.
Working in batches of 5, dip the tops of the cookies into the icing and return to the wire rack. Immediately top with the sprinkles. Allow the icing and sprinkles to set for about 20 minutes before serving or packaging.