For the topping: Grease a 9-inch ovenproof skillet (or 9-inch round cake pan) generously with the butter, being especially generous when buttering the bottom.
Sprinkle the brown sugar evenly over the bottom of the pan. Arrange the clementine slices on top, packing them in tightly, to completely cover the bottom. Set aside.
For the cake: In the bowl of a stand mixer fitted with the paddle attachment, cream the butter with the granulated sugar and brown sugar on medium speed until light and fluffy, 4 to 5 minutes.
Add the eggs one at a time, mixing well to combine after each addition. Add the vanilla extract and mix to combine.
Add the flour, polenta, baking powder, ginger and salt and mix to combine. With the mixer running on low speed, pour the buttermilk in a slow, steady stream. Mix well to combine and scrape the bowl well.
Pour the batter over the prepared clementine slices and gently spread into an even layer.
Bake until a toothpick inserted into the center comes out clean, 40 to 45 minutes. Let the cake cool for 10 minutes in the pan then invert onto a wire rack and let cool completely.