Line an 8-by-8-inch baking pan with aluminum foil and lightly grease with vegetable spray. Sprinkle the peanuts over the bottom of the pan. Put the milk and sugar in a medium, heavy-bottomed saucepan fitted with a candy thermometer. Boil over medium heat, occasionally stirring to keep from sticking. Cook until the mixture reaches 240 degrees F (soft ball). Remove from the heat and gently place the butter, peanut butter and vanilla in the pan, taking care not to disturb the syrup. Do not stir. Let the mixture cool down for 5 minutes, then beat vigorously with a spoon or the flat beater of a stationary mixer until dense and smooth. Turn the fudge into the prepared pan, spreading it evenly with a metal spatula. With a sharp knife, score into portions. Let set, 45 minutes then cut into 1 1/3-inch squares.