Preheat a grill to medium. Oil the grill grates well.
Slice the potatoes lengthwise about 3/4 inch thick. Place the potatoes in a bowl and add 2 tablespoons canola oil, the paprika and 2 teaspoons each salt and pepper. Toss to coat.
Place the potatoes on the grill, cover and cook 5 minutes, then rotate them 90 degrees F and cook 5 to 10 more minutes. Flip the potatoes and repeat on the other side. They should be slightly puffed and golden brown. Remove and allow to cool completely. Dice into 3/4-to-1-inch pieces.
In a large glass bowl, combine the diced potatoes, celery, red onion and cilantro. In a medium glass bowl, combine the sour cream, mayonnaise, mustard, 1 teaspoon salt, 2 teaspoons pepper, the capers and lemon juice. Adjust the salt as needed.
Fold the mayonnaise mixture into the potato mixture until coated. Cover and refrigerate 20 to 30 minutes. Stir and adjust the seasoning as needed.