Into a large saucepan, mix together milk, rice, sugar, vanilla bean and seeds, allspice and salt over medium-high heat. Let simmer for 45 minutes, stirring frequently and being careful not to scorch the milk. Once rice is tender and pudding has thickened turn off heat and remove the vanilla bean.
In a heavy skillet, melt butter and add brown sugar over low heat. Stir to melt the sugar. Add sliced bananas and continue to saute until they are soft and covered with the sugar and butter mixture.
Spoon pudding to serving bowls and top with the warm banana topping.