Make topping: In a well-seasoned 10-inch cast-iron skillet melt butter over moderate heat until foam subsides. Reduce heat to low. Sprinkle brown sugar evenly onto bottom of skillet and heat, undisturbed, 3 minutes (not all brown sugar will be melted). Remove skillet from heat.
Peel and core pears and halve lengthwise. Cut 3 pears halves lengthwise into 1/4-inch-thick wedges and arrange wedges, overlapping, around edge of skillet. Cut remaining pear half crosswise into 1/4-inch-thick slices and fan decoratively in center of skillet.
Preheat oven to 375 degrees F.
Make cake batter: Into a bowl sift together flours, baking powder, salt, and nutmeg. In another bowl with an electric mixer beat together butter and sugar until light and fluffy. Beat in vanilla and almond extract. Add eggs, 1 at a time, beating well after each addition. With mixer on low speed add milk, beating until just combined. Beat in flour mixture in 3 batches until just combined.
Spoon batter over topping in skillet, spreading evenly (be careful not to disturb topping), and bake in middle of oven 40 to 45 minutes, or until golden brown and springy to the touch. Cool cake in skillet on a rack 5 minutes. Run a thin knife around edge of skillet. Invert a plate over skillet and invert cake onto plate (keeping plate and skillet firmly pressed together). Carefully lift skillet off cake and replace any fruit that is stuck to bottom of skillet if necessary.
Serve cake warm or at room temperature with whipped cream.
Recipe courtesy of Gourmet Magazine