You can mix these colors with a standard box of red (r), yellow (y), blue (b) and green (g) food coloring. Mix 1 cup frosting with the drops indicated. (Don't be alarmed: 100 drops is only about 1 teaspoon.) If using store-bought white frosting, use the numbers in parentheses.
Make vanilla cupcakes.
Then make pure-white frosting. It's ideal for dyeing. Do not add vanilla extract as it will alter the color.