Preheat oven to 350 degrees F.
Grease and flour cupcake cups and place them in muffin tins. Using an electric mixer, cream the butter until fluffy. Add the sugar and continue to cream for approximately 6 to 8 minutes. Add the eggs 1 at a time, beating well after each addition. Add the flour and milk, alternating, into the creamed mixture, beginning and ending with the flour. Add the vanilla and continue to beat until well mixed. Divide the batter equally among the cupcake papers. Level the batter by dropping muffin pan on counter top to release the air bubbles. Bake for 25 to 35 minutes or until done. Cool in pan for 5 to 10 minutes.
Using a small bowl, combine the chocolate frosting and the peanut butter and blend well. When cupcakes have cooled, frost the tops with the peanut butter icing.