For the banana chips: Preheat the oven to 250 degrees F. Line a baking sheet with parchment paper and spray the paper with nonstick cooking spray.
Slice the bananas crosswise into about 1/8-inch-thick slices. Arrange the slices on the prepared baking sheet in a single layer. Sprinkle lightly with the cinnamon. Bake, carefully turning once halfway through, until golden brown and crisp, about 2 hours. Cool completely.
For the sundaes: Gently warm the guava jelly in a small saucepan with a splash of water to loosen. Stir in the lemon juice.
Serve the ice cream in bowls and spoon the guava coulis on top. Sprinkle with the banana chips.