In a medium bowl, sift together flour, cocoa powder, baking powder, and salt, and set aside. In the bowl of an electric mixer fitted with the paddle attachment, cream butter, vanilla, and sugar. Add eggs and milk, and mix until combined. Add reserved flour mixture, and mix on low speed until incorporated, scraping the sides of the bowl at least once. Divide the dough in half, and shape each half into a flat disk. Wrap each disk in plastic wrap, and chill until firm, about 1 hour.
Heat oven to 350 degrees F.
Roll dough out on a floured work surface, releasing dough with an offset spatula every few turns of the rolling pin. Roll dough to 1/8-inch thick. Cut dough using a variety of heart-shape cookie cutters 2 1/2 to 3 inches in diameter, making sure each cookie has a match to make a sandwich. Place hearts on a parchment–lined baking sheet; chill until firm, about 30 minutes. Remove from refrigerator; use a fork to prick the cookies with holes. Bake 12 to 15 minutes, or until just firm. Let cool on cookie sheet; transfer to a wire rack.
With the underside of half the cookies facing up, spoon softened strawberry ice cream about 1/2-inch thick to cover one side. Place matching cookie on top of ice cream, top-side facing out. Transfer immediately to freezer to harden; repeat with remaining ingredients. Serve directly from the freezer. Sandwiches can be kept stored in an airtight container for 3 to 4 days.
(c)2004 Martha Stewart Living Omnimedia, Inc. All Rights Reserved.