In a saucepan, stir together the sugars, salt, cocoa powder, milk, and heat until sugar is dissolved. Add the butter and cook to 238 degrees F on a candy thermometer.
Pour out of the pan into a mixing bowl to cool slightly and with a paddle attachment, stir in the peanut butter and vanilla. Continue beating to cool and stiffen the fudge until it starts to lighten and looses its gloss. Pour it into a greased pan and let it cool completely before cutting into 1 1/2-inch squares.