In a large bowl or the work bowl of an electric mixer, dissolve the yeast in the milk and add 1 tablespoon sugar and 1 cup of the flour. Mix well, cover bowl with tea towel and let stand in a warm place about 15 minutes for the yeast to activate. Place risen dough in mixer, set aside.
Take a large saute pan and heat it dry over medium-high heat. Pour in 1 cup of the olive oil and add rosemary. Add grapes, and raisins, mix together and allow to cool.
Begin to mix the dough, slowly, and mix another 4 cups of flour in to the yeast mixture with the dough hook attachment, then add a pinch of gray salt. Add half of the olive oil/grape mixture into dough. Knead until smooth with hands. Form the dough into a ball. Place some olive oil in large bowl to coat the sides. Place dough ball into bowl, and cover with tea towel. Cover with a damp towel and allow to rise in refrigerator until doubled in bulk about 45 minutes, or up to overnight for a fuller rise.
The next day, take the risen dough, and place in mixing bowl. With more olive oil from the grape/raisin mixture, press dough down into a flat disk with the heel of your hand. Using your fingertips, nudge the dough into a rectangle. The dough can be rolled but the pressure will produce heavier bread. Transfer the dough to the baking sheet and brush with the remaining 1/3 cup olive oil. Cover with tea towel, and let rise again until doubled, 30 to 40 minutes.
To bake, preheat oven to 425 degrees F.
Bake 15 minutes then remove from the oven and brush with the egg. Sprinkle with the remaining grape mixture then the remaining sugar. Top with lemon zest and finish with some coarse salt. Return to the oven and continue to bake until golden brown on top and crisp on the bottom, about 7 or 8 minutes. Let cool in the pan before cutting.