Position racks in the upper and lower thirds of the oven; preheat to 400 degrees F. Line 2 baking sheets with parchment paper. Mix the butter, 3 tablespoons sugar, the matcha, egg yolk, flour, milk and vanilla in a bowl with a rubber spatula until smooth. Mix in green food coloring. Add more food coloring, 1 drop at a time, until bright green.
Spread the remaining 1/3 cup sugar on a clean surface. Unfold the puff pastry and set it on top of the sugar, pressing gently to adhere. Flip and press to coat the other side. Roll out the pastry into a 12-inch square. Spread the matcha mixture on the pastry in a thin even layer, stopping about 1/4 inch from the edges. Using your fingers, roll one side of the pastry into the center, then roll in the opposite side to meet it. (If the pastry feels soft, freeze until firm, about 15 minutes.)
Trim the ends of the pastry roll with a knife, then cut crosswise into 1/2-inch-thick pieces. Arrange cut-side down about 2 inches apart on the prepared pans.
Bake until the cookies are browned and crisp around the edges, 14 to 16 minutes. Flip with a thin spatula, switch the position of the pans and bake until the other side is golden, 6 to 8 more minutes. Let cool completely on the pans.