In a medium bowl, add the cherries and the spiced rum. In another bowl add the pineapple and the 3/4 cup of rum. Cover and refrigerate for 8 hours or overnight.
Preheat oven to 350 degrees F.
In a large bowl, add 1 3/4 sticks butter and the sugar and cream together until pale yellow, about 2 minutes. Stir in the eggs 1 at a time. Add the vanilla extract and set aside.
In a separate bowl, combine the flour, salt, and baking powder. Add the dry ingredients into the wet ingredients and stir until a smooth batter forms.
In a 10-inch cast iron skillet, over medium heat, add the remaining 1/4 stick butter and heat until the butter has just melted. Add the brown sugar and using a slotted spoon, add the pineapple pieces, reserving the rum. Saute until they begin to caramelize, about 4 minutes. Add the cherries and the rum they soaked in. Remove the skillet from the heat and light the rum with a long kitchen match. When the flames subside, return the pan to the heat and cook until most of the alcohol has evaporated. Remove the skillet from the heat and set aside to cool 6 to 10 minutes before adding the batter. Remove half the fruit back into the reserved rum for garnishing later.
Add the batter to the skillet and put in the oven. Bake for 30 to 45 minutes or until a toothpick inserted into the center comes out clean. Remove from the oven and set aside to cool for about 5 minutes before unmolding. To unmold, run a small knife around the edge of the cake, take a large serving plate and put it over the cast iron skillet. Carefully flip over and let sit for a couple of minutes until it releases from the skillet.
Put the creme fraiche in a small bowl and stir in the zests. Cut the cake into wedges and serve with a dollop of creme fraiche and the reserved fruit.