Combine tomatoes, cheese, olives, garlic, 1 teaspoon oil, rosemary and pepper in a small bowl.
Preheat grill to medium.
Discard mushroom stems. Remove brown gills from the undersides of the caps using a spoon; discard gills. Mix the remaining 1 teaspoon oil, lemon juice and soy sauce in a small bowl. Brush the mixture over both sides of the caps.
Oil a grill rack (see Tip). Place the caps on the rack, stem sides down, cover and grill until soft, about 5 minutes per side. Remove from the grill and fill with the tomato mixture. Return to the grill, cover, and cook until the cheese is melted, about 3 minutes more.
To oil a grill rack: Oil a folded paper towel, hold it with tongs and rub it over the rack. (Do not use cooking spray on a hot grill.