Preheat the oven to 350 degrees F.
For the cookies: In a large mixing bowl, cream together the butter and sugars until light and fluffy. Add the pumpkin, vanilla and egg and mix well. Mix in the flour, baking powder, baking soda, cinnamon, salt and nutmeg. Drop by tablespoonfuls onto ungreased baking sheets. Bake for 10 minutes. Let cool.
For the frosting: In a small saucepan, combine the brown sugar and butter. Bring to a boil and cook until slightly thickened, about 1 minute. Let cool for 10 minutes. Slowly add the milk and whisk until smooth. Slowly whisk in the confectioners' sugar until the frosting reaches the desired consistency. Spread the frosting over the cookies and sprinkle with the pecans if using.