Add the butter, brown sugar, granulated sugar, maple syrup and salt to a medium saucepot. Turn the heat to medium-high and allow the sugar to caramelize until it becomes deep amber and begin to smoke slightly, about 8 minutes. Add the heavy cream and milk and stir until the caramel melts back down (it will firm up when the milk goes in). Reduce the heat to medium.
Whisk together the cornstarch, eggs and egg yolks in a medium bowl. Add a cup of the caramel to the eggs, mixing vigorously until combined, then add the egg mixture to the caramel and stir to temper. Mix constantly until it starts to bubble and thicken, about 2 minutes. Remove from the heat and add the vanilla, cinnamon and nutmeg; stir to combine. Pour the pudding into 4 serving cups and cover with plastic wrap. Refrigerate until the pudding is cool and set, 4 hours or up to overnight.
Dollop each serving with whipped cream and sprinkle with a pinch of flaky salt. Enjoy!