Put the potatoes in a big pot with enough water to cover by 1-inch. Season with salt and bring the water to a boil. Cook just until the potatoes are tender when pierced with a fork, about 25 minutes.
Drain the potatoes and return them to the uncovered pot off the heat. Let them sit until almost room temperature. (Cooling them in the warm pot will get rid of any excess water in the potatoes, and that's good.)
Meanwhile, cut the white parts off the ends of the celery stalks. Cut the stalks in half lengthwise, then across into 1/4-inch slices. Stir the celery, 3/4 teaspoon salt, and the remaining ingredients together in a serving bowl large enough to hold all the potatoes.
When they're cool, cut the potatoes into 1-inch pieces, add them to the bowl as you go. Stir gently until all the potatoes are coated with dressing. You can make the salad up to a couple of hours in advance. Keep covered at room temperature. Don't refrigerate or the potatoes will lose their rich, smooth texture.