Preheat oven to 350 degrees F. Lightly spray an 11 by 7-inch baking dish or a medium sized oval baking dish with cooking spray.
Layer half of graham crackers, chopped bananas, chocolate chips, and marshmallows into the baking dish. Repeat with remaining ingredients. Bake in for 10 to 12 minutes or until marshmallows have turned slightly golden and chocolate chips are melted.
In a small saucepan over medium-low heat, combine the milk and pumpkin pie spice. Once the milk is hot, whisk in peanut butter until well combined and smooth and let cook for another 3 minutes until thick. Drizzle sauce over top of baked S'more's.