Preheat the oven to 325 degrees F. Grease an 8 x 4-inch loaf pan and line the bottom with parchment or wax paper.
Place the raisins and sherry in a small saucepan and heat gently for 5 minutes so the raisins will absorb the sherry. Remove from the heat and stir in the cherries and 3 tablespoons of the ground almonds. Set aside.
In a separate bowl, cream the butter and sugar until fluffy. Beat in the egg yolks and almond extract, and stir in the remaining ground almonds. Fold in the raisins and cherries.
Beat the egg whites until they form stiff peaks and fold gently into the cherry batter with a large metal spoon.
Pour into the prepared pan and bake for about 1 hour, or until a skewer inserted in the middle comes out clean. Cool in the loaf pan. If desired, turn the cooled cake over, poke a few holes in the bottom, and drizzle it with a few tablespoons of sweet sherry. Store in an airtight container for up to 2 weeks.
*If you can't find ground almonds, you can grind blanched whole almonds in the food processor until they are like fine cornmeal.