Put the fish cubes in a bowl. Soak eight 8-inch wooden skewers in water for at least 30 minutes.
Meanwhile, make the pesto: Put the almonds and garlic in a food processor and pulse until finely chopped. Add the mint, parsley and lemon zest and pulse until finely ground. Transfer the mixture to a small bowl and whisk in the olive oil; loosen with 1 tablespoon water, if needed. Season the pesto with salt, pepper and the red pepper flakes, if using. Pour about one-third of the pesto over the swordfish and toss to evenly coat. Reserve the remaining pesto for serving. Refrigerate the swordfish for at least 30 minutes and up to 4 hours.
Preheat a grill to medium high. Skewer 3 or 4 chunks of swordfish on each skewer. Grill, turning once, until just cooked through, 2 to 3 minutes per side. Drizzle the skewers with the reserved pesto and serve with lemon wedges.