Make the topping: In a small bowl blend together butter, sugar, flour, cinnamon, and nutmeg until the mixture resembles coarse meal, and chill the topping while making the batter.
Make the batter: Preheat oven to 350 degrees F.
In a small bowl with an electric mixer, cream together butter and sugar and beat in the vanilla. In a small bowl stir together baking powder, flour, and salt, beat the flour mixture into the butter mixture alternately with the eggs, 1 at a time, beating well after each addition, and fold in the blueberries and the nectarines.
Spread the batter in a well-buttered 10 by 2-inch round cake pan or 2-quart baking pan. Sprinkle the topping evenly over it and bake the buckle in the middle of a preheated 350 degree oven for 45 to 50 minutes, or until a tester comes out clean and the topping is crisp and golden. Serve the buckle with whipped cream.