Put the potatoes and 2 teaspoons salt in a large pot and cover with cold water by about 2 inches. Bring to a boil over high heat, then reduce the heat and simmer until the potatoes are tender but not falling apart, about 5 minutes. Drain.
Meanwhile, stir the mayonnaise, horseradish, mustard, vinegar, celery salt and cayenne together in a large bowl with 1/2 teaspoon salt and a few grinds of black pepper. Stir in the hot potatoes and the celery, celery leaves and shallots. Let cool to room temperature before stirring in the parsley and eggs. Season with salt and pepper. Refrigerate until ready to serve.