With a sharp vegetable peeler, peel the peaches and set them aside. In a large nonreactive saucepan, combine the sugar, saffron, chile, allspice berries, bay leaves, ginger and water. Over medium-low heat, stir until the sugar has dissolved. Increase the heat and bring the mixture to a boil, then lower the heat and simmer for 10 minutes. Add the peaches and continue simmering for about 30 minutes, or until the peaches are tender but not mushy. Turn them occasionally so that all sides take the color of the saffron evenly. They are done when a toothpick inserted into the fruit will go halfway through easily. Transfer the peaches with a slotted spoon to individual plates or serving platter, let cool and refrigerate. Garnish with mint and lemon zest.