Preheat the oven to 375 degrees F. Butter a 2-quart shallow baking dish or eight 6-ounce ramekins.
Make the topping: Whisk the cornmeal, flour, brown sugar and salt in a bowl. Stir in the pistachios. Work in the butter with your fingers until evenly moistened.
Make the filling: Halve and pit the nectarines and plums (no need to peel). Slice 1/2 inch thick. Toss with the blackberries, granulated sugar and flour in a bowl.
Transfer the filling to the prepared dish or ramekins and dot with 2 tablespoons butter. Squeeze handfuls of the crumble mixture and scatter on top of the fruit. Bake until golden and bubbly, 40 to 45 minutes.
Let sit 10 minutes before serving. Top with whipped cream or ice cream, if desired.