Place the potatoes on a sheet pan, add the garlic, rosemary, lemon, olive oil, 2 teaspoons kosher salt, and 1 teaspoon pepper, and toss until everything is coated with oil. Spread out in one layer.
Roast for 50 to 60 minutes, turning the potatoes with a metal spatula every 20 minutes, until the potatoes are browned and crisp on the outside and tender inside. The garlic will be browned and sweet and the lemon slices will be caramelized. Discard the rosemary branches, sprinkle with fleur de sel, and serve hot.