For the gravy: In a heavy-bottomed saucepan, add flour and oil. Cook over medium heat for about 10 to 12 minutes whisking constantly until a peanut-brown color. Whisk in milk, salt, pepper, and sugar. Bring to a boil, continuing to whisk. Reduce heat and simmer for 15 minutes whisking frequently until sauce thickens enough to coat the back of a spoon (not too thick, but not too thin). Make gravy before you fry the chops.
For the lamb: Add 1-inch of oil in a straight-sided 3-inch deep pan. Preheat oil in frying pan to 365 degrees F. Mix salt and pepper with flour. Dredge the lamb chops in the seasoned flour. Dip lamb chops in buttermilk, then back into the flour. Slowly add chops to preheated oil, about 4 per batch. Cook 2 minutes per side for medium doneness. Drain on a rack.
Serve with gravy.