In both cases, you'd have to have the time period feel accurate and the characters fit their times.
If someone from 100 years ago walked into today, they'd be more than lost. They'd be terrified. They'd recognize iconic structures, say the Statue of Liberty and Ellis Island (I hope I'm not mixing dates up), but not much else in NY harbor. Fashion would be shocking, and embarrassingly so.
Someone going back in time would have more reference. They'd have maybe seen pictures or picture books. There would still be culture shock--no TV or computers, what! And they'd still be way out of place, but horses pulling vehicles wouldn't be strange. After all, they still do in Central Park.
There are many books with time travel elements: The Magic Tree House series, for example. Of course, not a single picture book comes to mind. There are a few TV shows for kids that have done this too. A Google search will turn them up. Add an age group to narrow the list.