import pbf to geopandas without filtering

[Eph97]

I was wondering if there was a way to import a pbf into geopandas using pyrosm without filtering the data. I'm dealing with the osm planet file and have been using osmctools to select the specific regions and data I need. I had been using osmnx to analyze the data until the regions I was looking at became a little too large for osmnx. But as a result I have existing python code I'd love to use. I just have been struggling to figure out if there's a way using pyrosm to simply read my prefiltered pbf file directly into a geopandas dataframe.

On a separate note, I love the goal of this project. I've been watching its evolution since this summer and am excited to finally maybe be able to use it in my research.

[HTenkanen]

Hi @Eph97! Hmm, interesting question! 🙂 So do I understand correctly that basically you would like to export everything, i.e all the features you see on a map at openstreetmap.org plus also the "hidden layers" that are not rendered on the map? Or what do you mean by reading pbf directly into geodataframe? Do you need to access all the elements, i.e nodes, ways and relations as "raw data"?

This is an interesting question. I believe it should be possible with the custom filtering (haven't tested yet), but the memory management might be a bottleneck.

EDIT: Ah right, so you are doing all the filtering yourself with the osmctools, so would only need to read the result from pbf?

[HTenkanen]

Would you have an example osm.pbf file that you have filtered, which I could use for testing?

[Eph97]

Hi @HTenkanen, exactly. I was able to get things working by just adding all the keys I was keeping with osmfilter using a custom filter such as:

custom_filter = {'amenity': True, 'boundary': True, 'historic': True, 'landuse': True, 'leisure': True, 'natural': True, 'tourism': True} (I'd already filtered out the values I didn't want so I figured just setting the keys that remained to true would work). This felt a little hack-y though and figured it would be better to try to not filter it at all if I'd already filtered out the data I don't need. Also, in the end I'll only really be looking at stuff like the area covered by greenspace relative to the area covered by roads in a given county etc so I only really care about the geometry column.

I attached a small pbf file (compressed so I could upload). It contains an example of the greenspace information I'm interested in for the city of houston. Because I'll ultimately be going county by county through the U.S. and also city by city through various commuting zones that I've constructed, it seemed to make sense to use the planet file and filter out this information using something like osmctools.

houston_greenspace.pbf.zip

thank you so much for your quick reply. I thought this was something other people might be interested in too so it was worth opening an issue.

[HTenkanen]

@Eph97 Thanks for your answer! Indeed, the approach you shared here was something that was on my mind as well. I will think about this a bit and see if there is a more straightforward way to accomplish the same without specifically specifying which keys to use. However, this is a bit special case (for more advanced user), as typically you do want to filter the data somehow.

But the least I can do, is to add a note about this into the documentation. Thanks for bringing it up! 🙏🏻 👍🏻

[Philipp-Otter]

Hello everyone,

I would love this feature as well. The filter could look something like {True: [True]} or {'True': [True]} as boolean is not accepted as osm feature.

Thanks for your work @HTenkanen.