Mix the peanut butter and sweetened condensed milk in a small saucepan over low heat. Gradually stir in the butter. Continue to cook, stirring, until the sauce is smooth. Remove from the heat and stir in the vanilla.
Let the sauce cool; it will start to thicken as it cools. When ready to serve, heat the sauce in the microwave for a few seconds so it is easy to drizzle on top of your dessert of choice.