Soak the herring in the milk for at least 12 hours or up to 24 to draw out a lot of the excess salt. Drain the herring, rinse, and pat dry. Cut off the fins and fillet the herring, leaving the skin on. Cut the fillets crosswise into 2-inch strips.
In a saucepan, combine the water, sugar, vinegar, allspice, mustard seeds, caraway seeds, and peppercorns. Bring the mixture to a boil, stirring to dissolve the sugar. Remove from the heat and cool the pickling mixture completely.
Arrange the herring fillets, onion, carrots, dill, bay leaves, and horseradish in a glass dish or funky Mason jar. Pour the cooled pickling mixture into the jar so that all the ingredients are completely covered with the liquid. Cover and refrigerate overnight or up to 2 days before serving.
