For the crisps: Fill a medium saucepan with 1/2 inch of vegetable oil. Heat over medium heat until the oil reaches 350 degrees F on a deep-fry thermometer. Slice the potatoes 1/8-inch thick, using a Japanese mandoline. Working in batches so as not to crowd the pan, fry the potatoes until crispy and golden brown, about 4 minutes per batch. With a spider or slotted spoon, remove the potatoes to a paper-towel-lined tray and drain well. Season with a few grinds of black pepper.
For the goat cheese: Using a rubber spatula, mix the goat cheese, mascarpone and orange zest together until well combined and smooth. Set aside at room temperature.
For the olive spread: Add the olives, capers and rosemary to a food processor. Pulse until finely chopped. Add the olive oil and pulse to puree; the finished spread should still have a little texture but be cohesive.
To assemble: Place a dollop of the goat cheese mixture on each potato crisp and top with a spoonful of the olive spread.