For the potatoes: Preheat the oven to 450 degrees F.
In a medium bowl, toss together the potatoes, salt, cayenne and olive oil. Place the potatoes cut-side down on a non-stick baking sheet and bake for 20 minutes, or until the potatoes are golden brown and cooked through.
For the aioli: While the potatoes are roasting, whisk together the mayonnaise, lemon juice, lemon zest, mustard and salt in a small bowl. Fold in the chopped parsley and set aside until ready to serve.
Serve the potatoes warm alongside the bowl of lemon-parsley aioli, for dipping.