Smash the ginger slices with the flat side of a chef's knife and transfer to a medium saucepan. Add the apple cider, orange zest, lemon zest, cinnamon sticks and cloves. Bring to a simmer over medium heat. Simmer 10 minutes, then strain through a fine-mesh sieve into a pitcher. Let cool to room temperature, then refrigerate until chilled, about 1 hour.
Poke holes in the apple wedges with a chopstick or skewer and insert a gummy worm into each. Add the cranberry-apple juice and whiskey, if using, to the punch. Serve over ice. Garnish with the apples.