In a medium saucepan, combine three cups of the milk, the pumpkin puree, cinnamon, allspice, cloves and ginger. Whisk to combine and then set over medium-low heat and bring to a gentle simmer.
Whisk together the sugar, cornstarch and salt in a large bowl. Whisk in the egg yolks and the remaining 1 cup milk.
Whisk half of the hot milk-pumpkin mixture into the egg mixture until smooth, then gradually whisk the egg-milk-pumpkin mixture back into the saucepan. Continue to cook, now over medium heat, whisking constantly, until the mixture boils. Continue to cook, whisking constantly, until it has thickened, 3 to 4 minutes. Remove from the heat and stir in the vanilla and butter.
To assemble, layer the cookies and pudding mixture in a serving dish or parfait glasses. Cover the pudding with plastic wrap and refrigerate until cold (at least 2 hours).