Preheat a grill to medium heat. Oil the grill grates well.
Slice the potatoes lengthwise into 3/4-inch thick slices. Place the potatoes in a bowl and add the oil, paprika and season with 2 teaspoons salt and 2 teaspoons pepper. Toss to coat.
Place the potatoes on the grill and cook until slightly puffed and nicely golden brown on both sides, 20 to 30 minutes, turning 90 degrees after 5 minutes and flipping and repeating on the other side. Remove and allow to cool completely. Dice into 3/4 to 1-inch pieces.
In a large glass bowl, add the diced potatoes, celery, red onions and cilantro.
In a medium glass bowl, add the sour cream, mayonnaise, mustard, 1 teaspoon salt, 2 teaspoons pepper, capers and lemon juice. Adjust the salt, as needed.
Fold the mayonnaise mixture into the potato mixture, cover and refrigerate for 20 to 30 minutes. Stir and adjust seasoning as needed.