Preheat a deep fryer or a deep pot of oil to 350 degrees F. In a medium bowl, sift together the flour, baking powder, a dash of salt and pepper, cinnamon, and the nutmeg, to taste.
In a small bowl, whisk together the egg and the buttermilk. Whisk into the flour mixture until smooth. Let rest for 30 minutes in the refrigerator.
Dip the banana chunks in the batter and fry until golden brown, about 4 minutes. Remove and drain on paper towels. Dust with powdered sugar and serve.