Preheat the oven to 350 degrees F.
In a large bowl, stir together the flour and baking powder. In a standing mixer with a paddle attachment, cream the butter with sugar until fluffy and light. Mix in the eggs, almond extract, and vanilla extract. Turn down the mixer and slowly add the flour mixture, a third at a time to make a stiff dough. Divide the dough into 4 pieces. Use 2 pieces to make cut out cookies, and the other half for the Russian Tea Cakes.
For the cut out cookies: On a lightly floured surface roll out a portion of cookie dough to 1/4-inch thick. Cut out shapes using cookie cutters. Place the cookies on lightly greased or nonstick cookie sheets. Bake until lightly golden, about 10 minutes, rotating cookie sheet halfway through the cooking time. Cool on pans about 5 minutes, and then transfer to wire racks and cool completely before decorating with colored icing.
In a small bowl, mix the sugar and water to form a thick, smooth icing. Stir in the food coloring to reach desired shade. Drop the icing onto cookies using a small teaspoon, and smooth with the back of the spoon. Make additional bowls for additional colors. Add sprinkles, if desired.;