Preheat oven to 350 degrees F. Spray a 9-by-5-inch loaf pan with nonstick spray or line with foil and spray with nonstick spray, for easy removal.
Mix brown sugar and 1 teaspoon cinnamon together in a bowl; set aside. Toss apples with 2 tablespoons granulated sugar and remaining teaspoon cinnamon in another bowl.
Beat butter and remaining 2/3 cup granulated sugar together in another medium bowl with an electric mixer until smooth and creamy, about 5 minutes. Beat in eggs, 1 at a time, until blended in; add vanilla extract.
Whisk flour and baking powder together in another bowl, then add to creamed butter mixture and beat until blended. Mix 1/2 cup milk into batter until smooth.
Pour half the batter into the prepared loaf pan; add half the chopped apple mixture.
Sprinkle half of the brown sugar/cinnamon mixture on top of apple layer.
Pour the remaining batter over apple layer and top with remaining chopped apples, then the remaining brown sugar/cinnamon mixture.
Lightly pat apples into batter; swirl brown sugar mixture through apples using a knife or spoon.
Bake until a toothpick inserted in the center of the loaf comes out clean, approximately 1 hour and 10 minutes.
Mix powdered sugar with remaining 1 to 3 tablespoons milk for a glaze, depending on preferred thickness.
Let the loaf cool for about 15 minutes, then drizzle with glaze.