Place cracker meal in a bowl and make a well in the center of the meal. Beat egg with milk and pour into well. Moisten cracker meal and combine liquid with meal using the tines of a fork. Add crab meat, crab boil seasoning, baking powder, parsley, lemon zest, celery, Worcestershire, hot pepper sauce, and mayonnaise to the bowl. Mix all ingredients well and form into 8 (3-inch) patties. Add 1/2-inch vegetable or canola oil to a skillet and heat to medium high temperature. Fry cakes 3 minutes on each side and drain on paper towel lined plate. Cook 4 cakes at a time, so that oil temp does not drop too low.
For red pepper sauce, combine drained red peppers in a food processor bowl with mayonnaise and chili sauce. Pulse/grind the ingredients into a smooth sauce. Transfer sauce to a small serving dish.