Preheat the oven to 375 degrees F. Line a baking sheet with parchment paper.
Using a melon baller, scoop out 2 cavities in each mushroom cap for the skull's eye sockets. Use a paring knife to cut a triangle for the nose area and to cut away lines in the stem for teeth. Place each mushroom upside-down (cap-side down) on the prepared baking sheet.
Bake 15 minutes for the medium mushrooms and 20 minutes for the large. Transfer to a paper towel-lined plate. Allow to sit and come to room temperature (the mushrooms will expel a lot of water at this point; use the paper towel to absorb the moisture).
Heat the balsamic vinegar in a small pot over medium-high heat until reduced by half. Cool to room temperature and drizzle over top of the mushroom skulls before serving.