Place lemon juice in a small non-reactive saucepan and reduce to 3 tablespoons. Place the butter in a medium bowl, add the lemon syrup and dill, then season with salt and pepper to taste. Mix well and scrape into a ramekin. Refrigerate for 1 hour.
Whisk together the zest, juice, oil, dill, and garlic, then season with salt and pepper to taste. Mix the pasta, cheese, peppers, olives, and scallions together in a bowl. Pour the vinaigrette over and mix to combine. Add the oregano and season with salt and pepper to taste.
Preheat grill to high. Toss the shrimp in some olive oil and season with salt and pepper to taste. Thread 5 shrimp on a skewer, bending the shrimp so that the skewer passes through it twice. Grill, turning once for 4 to 5 minutes. Remove the shrimp from the grill to a large platter and brush generously with Lemon-Dill Butter. Garnish with fresh dill.