Preheat the oven to 350 degrees F. Beat the butter and sugar with a mixer in a large bowl until the mixture lightens in color, 5 to 8 minutes. Slowly beat in the eggs and lemon zest. Sift 1 1/2 cups flour into the bowl. Add the baking powder and salt, gently folding to combine. Add the olive oil and milk, stirring to combine. Toss the raisins and about 1 cup grapes in a separate bowl and sift in the remaining 1 tablespoon flour. Add the fruit-flour mixture to the cake batter and stir only enough to blend.
Thoroughly grease the bottom and sides of a 9-inch springform pan or glass baking dish with some softened butter. Pour the batter into the pan and slide the pan into the center of the oven. Bake until a toothpick inserted in the center comes out clean, 40 to 45 minutes. Remove the cake from the oven and let sit until cool enough to touch, then detach the outside of the springform pan.
Whisk together the cream, confectioners' sugar and almond extract in a medium bowl until soft peaks form. Set aside.
Add the honey to a small saute pan and bring it to a simmer over low heat. Once it bubbles, thickens and darkens slightly, about 3 minutes, add the ginger and the remaining 1 cup grapes. Switch off the heat and allow the mixture to cool slightly. Serve slices of the cake with some of the grape-honey mixture and a dollop of the whipped cream.
Photograph by Con Poulos
Recipe courtesy Alex Guarnaschelli for Food Network Magazine