Lightly butter an 8-inch baking dish and set aside. Fill a large pot with enough water to come 2 inches up the sides of the pot. Set a steamer rack in the pot, cover, and bring the water to a boil over high heat. Add the squash and steam over medium heat until the squash is very tender, about 20 minutes. Transfer the squash to a food processor and blend until smooth and creamy. Season the squash to taste with salt and pepper.
Spoon half of the squash evenly over the prepared baking dish. Dollop half of the pesto all over the squash in the dish. Sprinkle half of the cheese over the squash. Repeat layering with the remaining squash, pesto, and cheese. Using a skewer, swirl the pesto decoratively into the squash. Dot the top with butter and bake until the gratin is heated through and golden brown around the edges, about 40 minutes.