Slice the pound cake (frozen is fine) into 1- to 1 1/2-inch slices. Next, with a biscuit cutter or cookie cutter that fits the bottom of the flower pot, cut the slices into rounds. (You can put the scraps in a zip-top freezer bag and use them at a later date.)
Into each of your (washed and dried) flower pots, drop a round of pound cake, lightly pressing to make sure they are all the way down. Hold a straw up against the side of a flowerpot and snip it so it's 1/2 to 1 inch shorter than the top. Then, just use that as a guide to cut the rest of the straws. Stick each straw into the center of each pot, anchoring it into the cake.
Now, grab whatever flavor ice cream you'd like to use, and either set it out on the counter to soften it OR nuke it for 20 seconds or so. It needs to be somewhat soft for this step. Just spoon it into the pots, working it around the straw, until it almost reaches the very tip top of the straw. Clean up the top edges of the pot with a damp paper towel. Now return the pots to the freezer until firm.
In a food processor, pulse the cookies until they resemble nice, rich topsoil. (You can also crush them in a zip-top bag with a rolling pin.) When they are nice and firm, take the pots out of the freezer. Place a couple of gummy worms on top of the ice cream, then spoon the dirt (crushed cookies) over the top. Be generous here! Place a fresh flower into the straw and serve immediately.
To clean the flower pots, scrub with warm soapy water to remove any dirt. Rinse with hot water. Let the pots dry thoroughly (it may take several hours).