Whisk the flour, sugar, baking powder and salt in a medium bowl. Stir in the melted butter and eggs until combined. Transfer one-third of the dough to a small bowl; add the cocoa powder and milk and stir with a wooden spoon until combined. Gently stir the cocoa dough back into the plain dough until it looks marbled.
Position racks in the upper and lower thirds of the oven and preheat to 350 degrees F. Line 2 baking sheets with parchment paper. Drop tablespoonfuls of dough about 1 inch apart on the prepared baking sheets; sprinkle generously with sugar.
Bake until the cookies are slightly puffed,6 to 7 minutes. Remove from the oven and make an indentation in the center of each cookie with the back of a teaspoon. Spoon about 1/2 teaspoon pudding into each indentation, then return to the oven, switching the position of the pans; continue baking until lightly golden and set, about 6 more minutes. Let cool 5 minutes on the baking sheets, then transfer to racks to cool completely.