Whisk together all-purpose flour, baking soda and salt in a medium bowl. Reserve.
Combine the butter, peanut butter and light brown sugar in another medium bowl. Beat with an electric mixer until light and fluffy, about 2 minutes. Add the flour mixture and blend until just combined. Shape the dough into a log: Place it onto a piece of plastic wrap about 18 inches long. Use your hands to pat the dough into a log about 7 inches long and 2 inches around. Roll the plastic wrap around the dough. Twist both ends of the wrap, like a candy, and tighten both sides until the dough log is compact and tightly wrapped (this might cause the dough to shrink down to a 6-inch log instead of a 7-inch log, but that is okay). Tuck the twisted ends under the dough and refrigerate until firm, at least 2 hours.
When ready to bake the cookies, preheat the oven to 350 degrees F and arrange 2 racks in the middle of the oven. Line 2 baking sheets with parchment paper. Remove the plastic wrap from the dough and use a serrated knife to slice the dough into 1/8-inch thick rounds. Place the rounds about 1 inch apart on the baking sheets, about 12 cookies per baking sheet. Press a halved peanut in to the center of each cookie. If the dough is still very cold, it might crack slightly around the peanut. Simply use your finger to lightly push the dough back together.
Bake the cookies until the edges are very lightly browned, 10-12 minutes. Halfway through cooking, rotate the baking sheets front to back and top to bottom.
Cool the cookies slightly on the baking sheets for 5 minutes then transfer to a cooling rack and cool completely.
Place the heavy cream and confectioner's sugar in to a large mixing bowl and use a hand mixer to whip until stiff peaks.
Line a 1.5 quart loaf pan with plastic wrap. The plastic wrap should come up the sides of the loaf pan and have at least a 2-inch over-hang. On the bottom of the loaf pan, lay 5 cookies face-side-down in a single layer. The cookies should touch but not over-lap. Spread 1 cup of whipped cream evenly over the cookies. Continue this layering process 3 more times. Place the last 4 cookies on the whipped cream and top the cookies with any of the remaining whipped cream. Fold the plastic wrap over the top of the loaf pan so that the entire top layer is covered. Refrigerate overnight.
For the Peanut Butter Glaze:
Right before serving, whisk together the milk, confectioner's sugar, and peanut butter until smooth in a mixing bowl.
To serve, flip the icebox cake out on to a platter and remove the plastic wrap. Drizzle the glaze all over the top of the icebox cake. Reserve extra peanut butter glaze to drizzle over individual slices. Serve immediately.