Preheat the oven to 350 degrees F. Brush two 9-inch-round cake pans with vegetable oil; dust with flour and tap out the excess.
Make the cake: Whisk the flour, cocoa powder, baking soda and salt in a medium bowl. Combine the granulated sugar, vegetable oil, buttermilk, eggs and vanilla in a large bowl; whisk until smooth. Whisk in the food coloring, then add the flour mixture and whisk until smooth.
Spread 3/4 cup batter in each of the prepared pans (you'll have a very thin layer of batter); set the remaining batter aside. Bake until a toothpick inserted into the center of a cake comes out clean and the top springs back when pressed, 16 to 18 minutes. Transfer to a rack and let cool about 5 minutes in the pans, then invert the cakes onto the rack to cool completely.
Meanwhile, cool the pans under cold water, then wipe clean. Brush with oil and dust with flour again. Repeat to make 2 more cake layers, using 3/4 cup batter for each, then repeat again. You will have 6 layers.
Make the frosting: Beat the cream cheese, butter and salt in a stand mixer fitted with the paddle attachment on medium-high speed until fluffy. Beat in the vanilla, then beat in the confectioners' sugar in three additions until smooth.
Put 1 cake layer on a serving plate and spread evenly with a scant 1 cup frosting. Repeat with the remaining 5 cake layers, then frost the top and sides of the cake with the remaining frosting in a thin layer (just enough so the nonpareils will stick). Refrigerate until the frosting is just set but still soft, about 10 minutes.
Put the cake (on the plate) on a rimmed baking sheet. Cover the top and sides completely with the nonpareils, reusing the ones that fall onto the baking sheet.