For the potatoes: Preheat the oven to 425 degrees F. Toss the potatoes with the oil, lemon juice and pepper in a medium bowl; season liberally with salt. Spread the potatoes on a baking sheet and bake until tender and the skins start to loosen from the insides, about 30 minutes.
Meanwhile, combine the butter, sour cream and lemon zest with a wooden spoon. Season with some salt and pepper.
When the potatoes are done, use a fork slice to slice halfway through the potatoes lengthwise, and push the ends together. This will make the insides fluff up. Top each potato with some sour cream butter and a 1/2 teaspoon or so of caviar and chives, and serve hot. (You can also cool the potatoes slightly before topping¿the sour cream butter won't melt¿They are delicious both ways!)
For a vegetarian version, substitute a few tablespoons rinsed, chopped brined capers for the caviar.