Preheat a grill to medium high and prepare for indirect cooking: For a gas grill, turn off one or two of the center burners. On a charcoal grill, bank the coals to the sides, leaving the center open. Grill the nectarines: Toss the nectarines with the melted butter and brown sugar in a medium bowl. Grill over direct heat until well marked, 2 to 3 minutes per side. Remove to a bowl.
Cover the grill and reduce the heat to medium or medium low (the temperature should be 350˚ F to 400˚ F). Prepare the cake: Butter a 10-inch cast-iron skillet. Whisk the flour, baking powder, baking soda and salt in a medium bowl. Beat the butter and 2/3 cup granulated sugar in a large bowl with a mixer on medium-high speed until fluffy, 2 to 3 minutes. Beat in the egg and almond extract until well combined. Reduce the mixer speed to low and beat in the flour mixture in two batches, alternating with the buttermilk.
Spread the batter in the prepared skillet. Arrange the nectarines on top, cut-side up. Place the skillet over indirect heat, cover the grill and cook, rotating the skillet halfway through, until the cake is golden brown and a toothpick inserted into the center comes out clean, 40 to 50 minutes. Let cool slightly.
Beat the creme fraiche and the remaining 1 tablespoon granulated sugar in a medium bowl with a mixer on medium-high speed until soft peaks form, about 2 minutes. Serve with the cake.