Pour the milk into a saucepan and scrape in the seeds from the vanilla pod. Add the pod to the pot, too, and judiciously grate in some nutmeg. Heat the milk to the boiling point. Remove from the heat, cover, and let sit 10 minutes to infuse.
Meanwhile, beat the egg yolks and sugar together in a bowl. When the milk has infused, whisk it into the eggs and return the whole thing to the saucepan. Place the pan back on the heat and whisk to thicken slightly, about 3 to 5 minutes. Do not allow the mixture to boil. Stir in the flower water, to taste. Strain the mixture into a jug, cool, and, chill.