Make the cupcakes: Preheat the oven to 350 degrees F and line a 12-cup muffin pan with nonstick liners. Whisk the flour, baking soda and salt in a medium bowl and set aside. Heat the milk in a small saucepan until hot but not boiling; pour over the cocoa powder in a large bowl and whisk until smooth. Let cool slightly. Whisk in the granulated sugar, vegetable oil, egg, vanilla, flour mixture and mint chocolates.
Divide the batter among the muffin cups, filling each about two-thirds of the way. Bake until the tops spring back when lightly pressed, 20 to 25 minutes. Transfer to a rack and let cool completely.
Meanwhile, make the frosting: Beat the melted butter and cocoa powder in a large bowl with a mixer on medium speed until well combined, about 2 minutes. Reduce the speed to low; add the confectioners' sugar, marshmallow cream and milk and beat until combined. Increase the speed to high and beat until fluffy, about 4 more minutes. Set aside at room temperature until thickened, about 1 hour.
Transfer the frosting to a pastry bag fitted with a star tip; pipe onto the cupcakes. Top with nonpareils and/or coarse sugar.