The van Cortlandt Museum (an 18th century stone house) is another gem. Take the A train to the end. It's right in Van Cortlandt Park, in northern Manhattan. I also love the New York Historical Society on the Upper West Side, across from Central Park and close to the Natural History Museum.
Steel, glass and modern skyscrapers are the part of New York City that is often portrayed in movies, books, and pop culture. You'd be surprised at the amount of lightning hidden between skyscrapers and the amount of outdoor fun New York has to offer. It was a working farm, the wartime headquarters of George Washington, a tavern, and the home of Eliza Jumel, a real estate magnate who became one of the richest women in New York. As for New York's hidden gems, the New York Federal Reserve Bank's golden vault literally fits the description.
It now floats above tiny Staple Street in Tribeca like a remnant of New York at the turn of the century. But what you do want to see in New York City's largest public library, known colloquially as the main branch, is its majestic Rose Main Reading Room, an impressive room two blocks in length. When the bridge was originally built in 1907, it had the functional purpose of connecting the New York Hospital with its horse-drawn ambulance garage, so that patients could be transported without having to go outside. You can't say you've discovered all the secrets of New York unless you've been to the Mystery Bookstore that belongs to an equally mysterious owner.
And if you're worried about crime in the subway, I can tell you that this New Yorker rides the subway all the time, as do her friends, neighbors and family, including two grandchildren (real New Yorkers, they love the subway and hate cars). But if you're looking for a more meditative treat for the eyes and ears, spend an afternoon wandering around the often-overlooked Conservatory Garden, one of the most spectacular and romantic public gardens in New York City. She was once part of the second general manager of the personal tobacco plantation in New Amsterdam in the 17th century and, later, she was involved in the first prison in the state of New York, the Newgate State Prison. If you're going to New York for the first time, keep your eyes peeled for enigmatic doors, mysterious buildings, historic streets, and houses that seem to be from another era.