Place an oven rack in the center of the oven. Preheat the oven to 350 degrees F. Line a 12 cup muffin tin with paper liners. Set aside.
For the filling: In a small, heavy saucepan, combine the caramel candies and heavy cream over low heat. Cook, stirring occasionally, until the candies have melted and the mixture is smooth, about 10 minutes. Set aside to cool slightly.
For the cupcakes: In a medium bowl, whisk together the flour, baking powder and salt.
In a stand mixer fitted with the paddle attachment, beat the granulated sugar and butter on high speed until light and fluffy, 2 to 3 minutes. Reduce the speed to medium and beat in the vanilla and eggs. With the machine running, in batches, gradually add the flour mixture to form a thick batter. Beat in the peanut butter.
Place 3 tablespoons of batter in the bottom of each cupcake liner. Top with 1 tablespoon of the caramel mixture. Add 1/4 cup of batter on top of the caramel mixture.
Bake for 23 to 25 minutes until a cake tester inserted into the center of the cupcakes comes out with moist crumbs. Cool for 5 minutes. Remove the cupcakes from the muffin tin and cool completely on a wire rack, about 1 hour (center of cupcakes will fall during cooling).
For the frosting: In a medium bowl, using an electric mixer, beat the butter until light and smooth. Beat in the powdered sugar and milk until smooth and creamy. Using a rubber spatula, fold in the peanut butter until swirls of peanut butter form throughout the frosting.
Place the frosting in a piping bag and top the cupcakes or use a small spatula to frost the cupcakes.