In a large bowl, mix together the pudding mixes, buttermilk, milk and vanilla. Fold in the whipped topping until incorporated. Fold in the pineapple and most of the orange segments, reserving some for garnish. Chill for at least 1 hour.
Crush, then stir in two-thirds of the cookies just before serving. Garnish with the reserved mandarin orange segments and remaining whole cookies.