Preheat an outdoor grill or grill pan. Grill the pound cake, turning once, until slightly charred. Divide the slices among 4 plates.
Brush the cut sides of the peaches with oil and place on the grill, cut-side down. Grill until golden brown, about 2 minutes. Turn the peaches over and grill until almost soft, about 1 minute more.
Cut the peaches into about 1/2-inch thick slices.
Place a scoop of ice cream on each grilled pound cake slice. Top each with some of the sliced peaches, drizzle with the raspberry sauce, and sprinkle with the almonds. Serve.
Place all the ingredients in a food processor and process until smooth. Strain into a small bowl. Sauce may be refrigerated for 1 day.