Certainly there has been plenty written about Paley Park, on 53rd Street between 5th and Madison Avenues. I'm not going to add to that material, but did want to post a photo in case readers are unfamiliar with it.
The park is beautifully simple; essentially a cube facing 53rd street, with ivy, rotational herbaceous plants and honey locusts. The wall of water at the back of the park - as every landscape architecture student quickly learns - was a brilliant gesture. The white noise it provides drowns out the sounds of city and profoundly adds to the sense of peace on has when sitting on the Bertoia chairs.
Right next door, you can also get a glimpse of NYC's own piece of the Berlin Wall and maybe snap a photo.
Paley Park occupies the former site of the Stork Club, a famous club in operation from before prohibition until the mid-60s. Honestly, I wouldn't have even noted this, had it not been for the fact that I'm catching up on Mad Men. The episode I watched last night included a party scene there.