Cook the olive oil and garlic cloves in an ovenproof skillet over moderate heat until the cloves begin to sizzle and caramelize lightly. Add the thyme sprigs and let them sizzle in the oil for about 30 seconds. Add desired amount of black pepper and the juice of the orange. Add the olives and stir until they are hot throughout, 2 to 3 minutes. Add the orange zest.
Transfer the skillet to the oven and bake, stirring occasionally, until the olives start to soften, 5 to 8 minutes. Serve warm.