Blend the basil, pine nuts, garlic, zests, juices, salt, and pepper in a food processor until the mixture is finely chopped. With the machine running, gradually add the olive oil until the mixture is smooth and creamy. Transfer to a bowl and stir in the Parmesan.
Place a grill pan over medium-high heat or preheat a gas or charcoal grill. Brush both sides of the swordfish fillets with olive oil and season with salt and pepper. Grill the swordfish about 3 to 4 minutes on each side for a 1-inch thick fillet.
Transfer the grilled swordfish to serving plates, top with the citrus pesto, and serve.