MAPHACK: Hacking NYC's Past with NYPL Labs & Friends
John Stokell used to make clocks by hand, and a few years back he got his big break. Two inventors came to him with a new networking technology and he wanted John to help make it possible. So John signed on, built the "Registers" at either end of the line, and when the inventors' new technology took the world by storm, John's artisanal clock business took off. The money was good, very good apparently. By data mining publicly available metadata, it's possible to see that Stokell was able to move from his home at 130 Leonard in SoHo (which doubled as his studio) over the river to 154 Livingston St. in downtown Brooklyn, while moving his studio to 24 Platt St in the Financial District.
This kind of thing happens every day: people move, throw off data exhaust, and those changes are picked up in databases and acted upon by government officials, marketers, and researchers. Except not every artisanal clockmaker gets to work with Samuel Morse and help build the first electric telegraph in 1843 and have their move and commute picked up by a database query over 150 years later. John Stokell was one of dozens of individuals whose similar moves and commutes could be tracked between 1839 and 1854 revealed through historical data mining techniques.
Building ways to find stories like John’s was just one of the many things we got to make at NYPL Labs earlier this month as we set about to invite some of our friends from inside and outside The Library to two days of hacking with some of the coolest historical geospatial datasets we've been able to find and see what we could build together with modern tools and interfaces and old data.
[Yes, that's the data we were hacking on. Go run with it. Make amazing things. Then please share with us what you've done.]
A little backstory: 5 years ago Matt Knutzen, our Geospatial Librarian, teamed up with a globe trotting collective of historian geo-hackers and together they built The Map Warper: an open source tool designed to turn scans of our historical maps into contemporary GIS data. Since then, we've been methodically churning through historical maps, stretching them so they fit our electronic maps of the world, then methodically tracing the outlines of building footprints. We slowly built up a big stockpile of old places in New York City that, while no longer existing as buildings, roads, and parks, have left their physical footprints as references and words in maps, books, newspapers, and tax records.
With several years of data under our belts we invited those globetrotting Geo-Hackers, Topomancy, out to visit, then we thought of the coolest folks we know hacking history in novel ways (technological and otherwise). We readied all the related data we could and opened our doors…
The next two days were kind of a blur. Folks from all over the city (also the world) came to spend time away from their day jobs thinking about and building novel ways to explore records and notions of place and time. Folks from organizations as diverse as The New York Times and ProPublica worked alongside scholars from Columbia and Yale, alongside fanatics of New York's literary past alongside world-class coders from places like Hacker School, with support from the exceptional NYPL Technology team whose Digital Collections API made much of our work possible.
Donald Tetto of Findery may have put it best when he asked "How far away from history are we in terms of space and time? We could be at a distance of 10 feet and hundreds of years at the same time." The measurement of distance of space and time seemed to open an intellectual wormhole that once opened, couldn't be closed.
So we plied everyone with lunch (protip: 39th St. between 5th and 6th Ave. has 2 incredible Szechuan Chinese restaurants that the Labs team swears by), distributed commemorative cryptic T-shirts (designed by Mauricio Giraldo), and let the creative chaos begin. We didn't want to structure this as a contest or competition, but instead a more collegial atmosphere where we were all able to contribute to the problems others were tackling without fear of jeopardizing a shot at "the big prize," so we wound up with a solid mix of NYPL staff from the Map Division, Labs, NYPL Technology, or Topomancy either as an integral member of many groups.
In addition to geographically visualizing the stories contained in New York's historical city directories (which was done by Andrew Hill, the senior scientist at Vizzuality, these were just a few more of the truly amazing things that were built:
- The ProPublica Nerds contingent (Sisi Wei, Jeff Larson, and Mike Tigas) teamed up with Ashley Williams of Code For America and The Flatiron School to build NYPL Time Traveler: a working Foursquare application that brings historical photos of almost every street corner in NYC to Foursquare check-ins. They built on a project that Dan Vanderkam, a Google engineer who previously made Old SF, is working on with the Maps Division to power the photo lookups (be on the lookout soon for that one!). You can read more about NYPL Time Traveler on the Foursquare blog or on VentureBeat, or better yet, start using it today.
- Mauricio Giraldo, NYPL Labs' interaction designer, created a system to substantially improve the rate of map digitization by automatically recognizing the spatial footprint of buildings from our insurance maps - a solution long discussed as a holy grail which will likely result making it way easier for anyone to help NYPL make spatial history more accessible.
- Jefferson Bailey from the Metropolitan Libraries Council worked on a tool to georeference places mentioned in Robert Caro's incredible biography of Robert Moses, The Power Broker, in place and time and at urban scale (something that Topomancy's Schuyler Erle began at global scale with GutenKarte back in 2008 and made possible for NYC by the NYC Chronology of Place).
- Linked Data Superhero Cory Harper began building normalized ways to look for neighborhoods by varying colloquial names in an upcoming NYPL historical data tool [that is not at all unlike this upcoming tool from the Library of Congress also built by Topomancy].
- Evan Sandhaus of the New York Times R&D Labs started work to normalize The Times's place name vocabulary for future analysis (and we found out some places The Times writes about more than one would expect).
- Doug Reside, NYPL's Digital Curator for the Performing Arts, and Labs' Matt Miller (who's also part of the Linked Jazz brain trust) built a tool to visualize the geospatial footprint of musical theatre performances and venues in NYC.
- Labs' Paul Beaudoin worked alongside Andrew Leung and Peter Rood from Literary Manhattan and Deb Boyer from Azavea to create an interactive way to browse The Castello Plan, a 1660 plan for Lower Manhattan (back when New York was New Amsterdam) by merging text and maps from The Stokes Iconography of Manhattan Island, an amazing / crazy 6-volume text documenting Manhattan land history.
- Tim Waters and Sanjay Bhangar from Topomancy were able to visualize the spatial footprint of progress on the Map Warper over time, plotting the locations of warped maps over time, providing us an incredible understanding of the scale of work done as part of the NYC Historical GIS project.
- Peter Leonard, a Digital Humanities Librarian at Yale, in addition to wonderfully documenting the event on Flickr, built a tool to enable people to look up reassigned streets and addresses in Detroit, an incredibly useful prototype of a tool that could be used in any city with radical street address changes (I'm looking at you, Queens).
- And our pal Aaron Cope came down from the Cooper-Hewitt Labs and gave a mind expanding talk about the state of museums and libraries on the web, the state of the art geospatial technology (and why it hasn't moved much further than when he came up with lots of it at Flickr back in '07), and "archive realism".
This is just the start of what's going to be possible with historical geospatial data. The Open Historical Maps project is building a global collective of these kinds of materials that'll form the building blocks of this kind of work in the near future and we're speeding up our data production.
It's also the start of these kind of hack days at NYPL. In two days, we were able to clearly see a future where we can come together with the public as peers and tackle really awesome problems for the public good. We can promise there will be more, and we hope you'll be able to join us for one of them.
All photos in this post copyright Peter Leonard, 2013.
