Toss together the peaches, raspberries and brown sugar. Set aside to macerate.

Preheat the grill. Slice the bread. Arrange on a baking sheet and toast in the oven until light golden brown, about 10 minutes. Watch carefully to make sure they do not brown too much; set aside to cool.

In a saucepan, heat the half-and-half, cream, salt and vanilla bean over medium heat, stirring occasionally to make sure the mixture doesn't burn or stick to the bottom of the pan. When the cream mixture reaches a fast simmer (do not let it boil), turn off the heat. Set aside to infuse 10 to 15 minutes. In a large mixing bowl, whisk the eggs and granulated sugar together. Whisking constantly, gradually add the hot cream mixture. Strain into a large shallow bowl to smooth the mixture and remove the vanilla bean. Dip the toasted bread slices into the custard, and let soak until absorbed. Fold the mixture occasionally to ensure even soaking. Transfer the slices to the grill and cook about 5 minutes on each side. Serve warm or chilled, dusted with confectioners' sugar.