Preheat the oven to 350 degrees F.
For the mashed sweet potatoes: Cover the sweet potatoes with cold water in a medium saucepan and simmer over medium heat until fork-tender, 12 to 15 minutes. Drain off the excess water and transfer the potatoes to the bowl of a stand mixer fitted with the paddle attachment. Add the milk, cardamom and 4 tablespoons of the brown sugar and mix on medium until smooth, 3 to 4 minutes. Taste and add more brown sugar if desired. Set aside to cool.
Spray a 9-by-13-inch baking dish with cooking spray.
For the cake layer: In a clean bowl of a stand mixer fitted with the paddle attachment, cream together the egg and brown sugar until fluffy and thick, about 4 minutes. Add the yogurt, oil, vanilla extract and orange zest and mix until well combined. Add the flour, baking powder, baking soda and salt. Mix until the batter just comes together; it will be thick. Spread the batter into the bottom of the greased baking dish. Cover the batter evenly with the sweet potato mixture.
For the crumble: Combine the flour, brown sugar and melted butter in a small bowl. Mix with your hands until the mixture forms a rough crumbly texture. Sprinkle over the sweet potato and bake until the bottom is set and the crumb topping is golden brown, about 30 minutes.