Preheat the oven to 350 degrees F. Grease two 8-inch baking dishes with the nonstick cooking spray and line them with parchment paper, allowing 1-inch wings to hang over the edges; set aside.
In a large bowl, whisk together the flour, brown sugar, granulated sugar, cinnamon, salt and baking powder. In a medium bowl, whisk together the butter, vanilla, eggs and 3/4 cup peanut butter. Add the wet ingredients to the dry ingredients and stir with a wooden spoon or spatula to combine. Divide the mixture between the prepared baking dishes and spread out evenly with a spatula or your hands. Bake until the centers are just set, begin checking for doneness at 23 minutes. Cool in the pans completely.
While the blondies cool, allow the ice cream to soften at room temperature to a spreadable consistency. Using an offset spatula, spread the ice cream evenly over one pan of the blondies while it's still in the pan. Drizzle on the remaining 1/2 cup peanut butter and use the spatula to swirl it into the ice cream. Place the other blondie on top of the ice cream and freeze until the ice cream is firm, about 4 hours.
Using the parchment wings, lift the whole thing out of the pan and onto a cutting board. Use a sharp knife to cut to trim the edges (if desired, this isn't necessary) and cut it into 1 1/2- to 2-inch squares. Wrap individually in plastic wrap or place in an airtight container and store in the freezer.