Drain the green and black olives from the brine or oil that they're packed in and place them in a medium bowl. Add the orange zest, garlic, fennel seeds, thyme leaves, red pepper flakes, 1/2 teaspoon salt and 1/2 teaspoon black pepper. Pour the olive oil over the mixture, add the thyme sprigs and toss to combine.
Transfer the mixture, including the olive oil, to a medium (10-inch) sauté pan. Heat over medium heat until the oil begins to sizzle. Lower the heat and sauté for 4 to 5 minutes, stirring occasionally, until the olives and garlic are heated through and fragrant. Serve warm right from the pan or transfer to a serving dish. Offer a small dish for the pits.
This dish depends on the flavor and texture of the olives. Use high-quality olives with pits rather than the canned pitted ones.