Blend the condensed milk, evaporated milk and lime juice in a blender until smooth.
Spoon 1 cup of the lime cream into the bottom of a 6-cup glass trifle dish. Sprinkle with 1/3 of the lime zest. Then arrange 1/3 of the crackers over the cream, breaking them as needed to form a single layer. Repeat to make 2 more layers, ending with a layer of cream and a sprinkle of lime zest. Cover tightly with plastic wrap and chill until set, at least 2 hours, or overnight.
To serve, thinly slice 1 lime and arrange the lime slices in an overlapping layer on top as garnish, if desired.