Puree the red and yellow tomatoes in a blender until smooth, about 1 minute. Strain the puree through a fine-mesh strainer into a bowl and discard the solids. Stir in the vodka, lime and lemon juices and horseradish. Add the hot sauce and season with salt and pepper.
Fill a tall glass halfway with ice. Thread some baby tomatoes on a wooden skewer. Pour the tomato mixture over the ice, garnish with the skewered tomatoes and serve.
This drink can be made with any combination of green, red or yellow heirloom tomatoes for variations in color. If using green tomatoes, you will need to increase the amounts of the liquid ingredients.