For the frozen syrup: Using a paring knife, split the vanilla bean in half and scrape out the seeds. Place the seeds and the empty pod in a medium saucepan. Add the water and whisk until the vanilla seeds have separated. Add the sugar, lemon juice, lemon zest, lime juice, and lime zest. Bring the mixture to a boil over medium heat, stirring occasionally until the sugar has dissolved. Simmer for 5 minutes, remove the pan from the heat, and cool completely, about 20 minutes. Remove the vanilla pod and discard. Pour into an 8-by-8-inch glass baking dish and freeze for at least 10 hours or preferably overnight until frozen (mixture will still be slightly soft).
For the topping: In a small bowl, mix together the creme fraiche and sugar. Set aside.
To serve: Divide the mango between 4 dessert glasses. Using a spoon or an ice cream scoop, spoon a layer of the frozen syrup on top. Divide the pineapple and place on top. Spoon another layer of frozen syrup on top. Drizzle the top of the parfaits with the topping and serve immediately.