For the cupcakes: Preheat the oven to 350 degrees F. Fill a 12-count muffin tin with liners and set aside.
With an electric mixer, cream together the sugar and butter until fluffy, about 5 minutes. Add the eggs, one at a time, beating well after each addition. Sift the flour, baking powder and salt together, then add alternately with the milk, beginning and ending with flour mixture. Add the vanilla and mix until smooth.
Divide the batter evenly among the prepared muffin tin cups, filling about three-quarters of the way up. Bake for 18 to 20 minutes, or until a toothpick comes out clean. Turn out cupcakes onto a cooling rack to cool completely before frosting.
For the frosting: With an electric mixer, beat the butter and peanut butter together until fluffy, about 5 minutes. Slowly add 1 1/2 cups of the confectioner's sugar, the milk, and the vanilla. Beat in the remaining 1 1/2 cups confectioner's sugar. Mix in additional milk, 1 tablespoon at a time, until the frosting is a creamy, spreadable consistency.
Generously frost the cupcakes. Garnish with sprinkles.
For an even richer frosting, use heavy whipping cream instead of milk. For some crunch, try replacing half of the creamy peanut butter with crunchy.