Prepare the fish: Whisk the garlic, olive oil, orange juice, vinegar, honey, thyme, cumin seeds, 1/2 teaspoon salt and a few grinds of pepper in a large bowl. Add the fish and turn to coat. Cover with plastic wrap and refrigerate 1 hour.
Meanwhile, make the potato salad: Put the potatoes in a pot and cover with cold water; season with salt. Bring to a boil and cook over medium-high heat until tender, about 20 minutes; drain. Toast the cumin seeds in a medium skillet over medium heat, about1 minute; transfer to a large bowl. Add the olive oil and chorizo to the skillet and cook until the chorizo just begins to sizzle, about 3 minutes. Remove the chorizo with a slotted spoon and drain on paper towels. Remove the skillet from the heat; reserve the drippings.
Add the vinegar and herbs to the bowl with the cumin, then whisk in the chorizo drippings. Add the celery, piquillo peppers, potatoes and chorizo; toss. Season with salt and pepper.
Preheat a grill to high; brush the grates lightly with vegetable oil. Place the swordfish on the grill and brush with some of the marinade from the bowl. Grill until marked on the bottom, about 5 minutes. Flip and continue grilling until just cooked through, about 4 more minutes. Divide among plates and drizzle with olive oil. Serve with the potato salad.