Spray a rimmed baking sheet with nonstick cooking spray. Spread the marshmallows on the baking sheet and broil until deep brown, about 30 seconds. Remove from the oven and set aside for 5 minutes to cool.
In a stand mixer with the paddle attachment, beat the ice cream on medium-low speed until just soft. Add the toasted marshmallows and the chocolate chips and mix on low until just combined. Working quickly, scoop about 3 tablespoons of the ice cream mixture onto half of the graham crackers. Top with the remaining graham crackers and place in the freezer. Freeze until firm, at least 2 hours.
If the ice cream mixture is too soft to scoop onto the graham crackers, return the ice cream to the freezer for 30 minutes to firm up slightly before proceeding.