Scrub the potatoes under cold water to remove any dirt. Then, completely dry the potatoes.
Heat your grill to 375 degrees F. Wrap each potato in foil and place over indirect heat for 1 hour. Then, remove them from the grill.
Once the potatoes are cool enough to handle, cut the top third off and remove all but a quarter-inch of the flesh of each potato. Stir the flesh from all 4 potatoes in a bowl with 1 stick butter, sour cream, sliced green onion, and buttermilk, until thoroughly combined and a mashed-like consistency forms. Then, season with salt, pepper, and garlic powder to taste.
Brush the outer skins of each potato with olive oil and sprinkle them with salt. Place the oiled and seasoned potato shells on a sheet pan and evenly fill each potato with the mashed-potato mixture. Then, evenly top each potato with shredded cheddar and bacon crumbles.
Place the filled potatoes directly on the grates of the grill over direct heat. Slightly press down on each potato to get a flat bottom for extra contact. Cook until skins are crisp and grill marks form, about 8-10 minutes. The insides of the potatoes should be warm and the cheese fully melted. Then, remove from grill and dig in.