Preheat the oven to 325 degrees F and coat a 12-cup muffin pan with cooking spray. Lightly flour the pan, tapping out the excess. Sift the flour, baking powder, baking soda and salt through a fine-mesh sieve into a bowl; set aside.
Microwave the butter with 1/2 cup water in a large microwave-safe bowl until melted. Whisk in the cocoa powder, brown sugar and granulated sugar until smooth (if the mixture is too thick, microwave 20 more seconds). Let cool.
Whisk the sour cream, egg and vanilla into the cocoa mixture until smooth. Whisk in the flour mixture until just combined (do not overmix). Divide the batter among the prepared muffin cups, filling them halfway. Bake the cupcakes until a toothpick inserted into the centers comes out clean, 12 to 15 minutes. Let cool 15 minutes in the pan, then transfer to a rack to cool completely.
Using a paring knife, cut a circle into the top of each cupcake, about 1/4 inch in from the edge and 1 inch deep; pry out the cake. Transfer the cupcakes to a baking sheet and freeze at least 30 minutes.
Working quickly, fill each cupcake with a small scoop of vanilla ice cream. Top each with a larger scoop of strawberry ice cream, gently pressing so the strawberry ice cream is flush against the vanilla; freeze until firm, 20 minutes.
Pour the chocolate shell topping into a small bowl. Dip the tops of the ice cream cupcakes into the chocolate; freeze until serving.