geopipes

[joehand]

From @missinglink on July 21, 2014 21:29

Hey @maxogden, team.

As we discussed the other day at nodeconf in London, I'm currently leading a project at mapzen to create a set of curated stream libraries for working specifically with geographic datasets.

My personal interest in creating the libraries is to facilitate importing of large amounts of POIs in to elasticsearch for our opensource/opendata geosearch engine called Pelias.

However, outside that; it'd be great to have a set of stream libraries for dealing with these large geo datasets which can empower other developers to build their own geo pipelines.

I've just created the geopipes organisation on github today and I added an elasticsearch backend that should be compatible with dat, if you're interested in that sort of thing. https://github.com/geopipes/elasticsearch-backend

disclaimer: I only wrote that this morning, it has pretty full test coverage but I haven't given it a thorough beating yet.

Over the next couple of weeks I will add the repos for openstreetmap, geonames and quattroshapes as well as streaming parsers for osm pbf and shapefiles. (mostly written by others)

max asked me to open a ticket to discuss if we can find some way of working together and to make you aware of what we're working on.

Copied from original issue: maxogden/dat#140