Whisk the sugar, salt and yolks in a heavy medium stainless steel or enamel saucepan. Gradually whisk in the cream. Then whisk in the milk, buttermilk and citrus zests. Using a wooden spoon, stir the custard constantly over medium heat until it thickens enough to coat the back of the spoon and a thermometer registers between 180 and 185 degrees F, 8 to 10 minutes.
Immediately scrape the custard into a 4-cup measuring cup; mix in the rum. Refrigerate uncovered until cold, about 3 hours, stirring occasionally to prevent a skin from forming. Cover until ready to freeze (can be made 1 day in advance).
Freeze the custard in an ice cream machine according to manufacturer's instructions.
While the custard is freezing in the ice cream machine, peel and trim the ends from the zested orange. Using a paring knife, cut along the membrane on both sides of each segment. Free the segments onto a cutting board. Cut the segments into 1/2-inch pieces. During the last 10 minutes of freezing time, add the orange segments.
Transfer to a container; cover and freeze until ready to serve.