Remove all the fat and sinew from the veal loin and reserve them. Rub the loin with oil and season with salt and pepper.
Heat a saute pan over medium high heat. Add the veal loin and sear until golden brown on all sides, about 2 to 3 minutes per side. Remove the loin from the pan.
Place fat and trimmings in the bottom of a roasting pan and set the loin on top. Roast for 45 minutes. The roast should still be pink inside, not well done. Transfer it to a warm platter.
While the meat is roasting, peel the skins of the oranges and lemons, and cut into julienne. Place in a small saucepan with the sugar and enough water to cover. Boil until the sugar begins to caramelize. Pour the julienne into a strainer and rinse briefly under water. Set aside.
Squeeze the juice from the oranges. Remove fat and trimmings from the roasting pan and pour off any fat that has accumulated. Deglaze with the port and add the veal stock and orange juice. Simmer 15 minutes, strain, and reduce until only 1 cup remains. Whisk in the butter, 1 piece at a time, to slightly thicken the sauce. Reserve and keep warm.
Place a slice of the roast on a warm plate. Cover with sauce and sprinkle with the julienne of orange and lemon peels. Serve immediately.