Bring 4 cups water and the almond milk to a boil. Remove from the heat and add the tea bags. Let steep for 10 minutes. While the mixture is still warm, stir in the spiced rum and agave.
While the tea is steeping, whip the cream to soft peaks.
Remove the tea bags and ladle the spiked chai into serving cups. Top with a dollop of whipped cream and a dusting of ground cinnamon.