Add the shrimp to boiling, salted water and cook 2 to 3 minutes (cook too long and they'll get chewy, then mushy). Boil the diced potatoes separately or in the shrimp water until tender but still firm, 10 to 15 minutes.
Meanwhile, boil the eggs until hard, about 10 to 15 minutes. Allow all to cool while you dice the peppers and celery.
Dice the eggs and combine all of the remaining ingredients, adding the mayonnaise last, so you can control the texture of the salad. Chill and serve over a bed of lettuce or other fresh greens.