Press the rice cereal treat mixture evenly into a 13 by 9 by 2-inch pan coated with cooking spray. When cooled, cut into 12 pieces by cutting into thirds lengthwise and fourths crosswise.
Heat milk in a saucepan over low heat. Stir in sugar until it dissolves. Stirring constantly, add caramel topping until all ingredients are mixed and becomes a sauce. The sauce will be a consistency into which you will be able to fold in the bananas without mashing them, but is still thick. If necessary, use instant flour to thicken. Slice bananas into the sauce and fold in gently.
Spoon bananas and caramel sauce over rice cereal treats. Top with mini-marshmallows and a scoop of vanilla ice cream.