Cooking notes: Try using one of your favorite extra virgin or infused olive oils for added punch. Entertaining notes: These are best cooked way ahead and then reheated in a 400 degree F oven for about 15 minutes. If preparing for a larger group, simply multiply the recipe to suit your needs and cook in a nonstick rectangular baking pan. When cool, cut into serving size pieces and re-heat.
Cut the potatoes crosswise into 1/8-inch thick slices and place them in a large bowl. Heat the oil in a 10-inch nonstick ovenproof skillet over high heat. When the oil is hot, add the rosemary and olives. Cook for 10 seconds. Immediately remove the skillet from the heat and pour the mixture over the potatoes. Add the salt and pepper. Gently stir with a spoon to evenly coat the potatoes.
In the same 10-inch nonstick skillet, starting in the center of the skillet, arrange about 1/3 of the potatoes slices in a single layer so they form concentric circles that slightly overlap. Pick out about half the olives and sprinkle them and half of the lemon zest over the first layer of potatoes. Create a second layer of overlapping potato slices. Sprinkle the remaining olives and lemon zest over the second layer. Then create a third layer of potatoes. Finally, pour the oil from the bowl over the potatoes. Put the pan over high heat and cook until the oil sizzles and bubbles, about 1 minute. Place the pan in the oven and bake for 20 minutes, use a spatula to press down the top layer of potato slices. Continue to bake until the tip of a knife slides easily through the potatoes, about 20 minutes more. Remove the pan from the oven and pour off the excess oil. Allow the tart to sit at room temperature for at least 10 minutes.
Run a spatula along the outer edge of the tart to release any potatoes stuck to the side of the skillet. Place a dinner plate larger than the skillet on top of the skillet, with the serving side facing down. Invert the skillet and plate to transfer the tart to the plate. Cut the tart into wedges. Serve warm.