Preheat a grill pan over medium-high heat.
Toss the apples with some vegetable oil and add to the grill. Grill the apples on all sides, about 3 minutes. Once the apples are grilled, remove from the pan and roughly chop. Add the chopped apples to a saucepan and turn the heat to medium-low. Add the sugar, brandy, butter, vanilla extract and salt. Once the butter has melted, add the apple cider. Stir to combine the ingredients and continue to cook until the apples melt into a thickened sauce, thinning with apple cider as needed, about 10 minutes.
Brush the brioche with some oil and sprinkle with sugar. Place onto the grill pan and toast on both sides. Remove from the grill and slice diagonally.
Transfer to shallow serving bowls. Serve immediately with the apple caramel sauce and a generous scoop of vanilla ice cream.