ENH: Add possibility to export graph to GeoJSON

[HTenkanen]

[bramson]

Along these lines, I tried to export the networkX graph as a JSON as I usual do, but it threw a "TypeError: Object of type 'CRS' is not JSON serializable". Obviously, I need to use a GeoJSON instead of an ordinary JSON, but I'm having trouble locating instructions for exporting to GeoJSON.

Although the GeoPandas can be used for this purpose, the GeoJSON export ability would allow the results to be easily visualized with Kepler.gl, Tableau, etc.

Also, I noticed the NetworkX graphs seem to be generated from a geopandas intermediary, because nodes/edges without a property in OSM have {"key" : None} in the graph. In order to save memory, I will filter attributes with value == None, and perhaps this feature should also be added as an option in the to_graph function.

[HTenkanen]

@bramson Thanks for pointing out this! 🙏 Among many other issues! 🙂 I will take a look. Currently the CRS is saved as a pyproj CRS object which indeed cannot be serialized with JSON. I think this should be an easy oneline fix in the code.

[HTenkanen]

@bramson In terms of the extra "key" column, it relates to compatibility with OSMnx library. You can disable this behavior by specifying osmnx_compatible=False. Take a look at the Note in this section of the docs (still under develop branch): https://pyrosm.readthedocs.io/en/develop/basics.html#export-to-networkx-osmnx

[HTenkanen]

@bramson Could you still provide an example how you dump the JSON graph so that I could add a test for it?

[HTenkanen]

@bramson And do I now understand correctly that you would like to export the NetworkX graph (with both nodes and edges) as a GeoJSON? Or are edges enough? I would need to understand a bit more what you aim to do with the output to understand what would be the best way forward with this.

After a bit of thinking, I think the most straightforward way to save the edges is by using geopandas. If you need to export the directed graph (with the largest strongly connected component), you could do something like following:

from pyrosm import OSM, get_data
from pyrosm.graphs import get_directed_edges
from pyrosm.graph_connectivity import get_connected_edges

osm = OSM(get_data("helsinki_pbf"))
nodes, edges = osm.get_network(nodes=True)
nodes, edges = get_directed_edges(nodes, edges)
# (optional) Get connected edges 
nodes, connected = get_connected_edges(nodes, edges)
# Save to GeoJSON
connected.to_file("edges.geojson", driver="GeoJSON")

Note: please update to the latest version from master again before trying this ^.

[bramson]

1. I will try the osmnx_compatible=False setting, and see if it is equivalent to what I expected.

2a. Yes, I would like to output both nodes and edges of a NetworkX graph as a GeoJSON (for Kepler.gl visualization). Both nodes and edges have properties that I would like to visualize with different colors.

2b. Up to now, I've imported nodes and edges into Kepler separately from CSVs generated by geopandas, but uploading a single GeoJSON is more efficient (depending on the sparseness of properties, it may or may not use less memory than CSVs, but it is certainly an easier workflow to do one file with all the info).

3. My usual methods for saving graphs as JSONs:

class MyEncoder(json.JSONEncoder):
    def default(self, obj):
        if isinstance(obj, np.integer):
            return int(obj)
        elif isinstance(obj, np.float):
            return float(obj)
        elif isinstance(obj, np.ndarray):
            return obj.tolist()
        else:
            return super(MyEncoder, self).default(obj)

def writeJSONGraph(graphData, filePathName):
    with codecs.open(filePathName, 'w', encoding="utf-8-sig") as jsonFile:
        jsonFile.write(json.dumps(nx.json_graph.node_link_data(graphData), cls = MyEncoder))

[HTenkanen]

@bramson Thanks for sharing your ideas! 👍 Being able to export networkx graph as GeoJSON is certainly a good idea. However, I think that the correct place for implementing such a feature would actually be to networkx itself. But I think it would be easy to test the approach here.

Would you be interested in testing out / writing a PR for such a feature as you already seem to have many of the pieces in place? I think your approach with MyEncoder looks promising, and if I understand the functionality of it correctly, it would basically require adding new checks / converters for different kind of geometries to this encoder to convert geometries to valid GeoJSON objects. Probably a good place to start would be to check how the GeoJSON writing is implemented in Geopandas.

[bramson]

Yes, I think you are correct that the conversion should really be a part of NetworkX. I actually searched the interwebs for how to export NetworkX to GeoJSON, but it seems too niche to have already been asked.

For me, I need to learn more about GeoJSON to know where to start, and to determine whether the encoder approach could handle it. My thinking is that we could first use something like the above to create the base JSON, then add the Geo info to that, but until I look into it more I don't know if that's viable.

Alternatively, just for the sake of the output, I could do the NetworkX -> GeoPandas -> GeoJSON -> remove {k:None} -> export approach. Clearly not efficient for the computer, but still easier for me to create visualizations.

One thing to note is that the NetworkX that I want to convert isn't the one directly pulled from OSM. For example, I will pull the walking network and the train stations, then augment the network by connecting station nodes to the walking network, then I'll want to save that Network. So it really, really does seem like NetworkX issue and not directly related to your package.

That said, GeoJSON is a common format for geospatial networks, so supporting it as an export format would be great for pyrosm.

[HTenkanen]

@bramson Yes, supporting the export of geospatial network from nx is definitely something that should be implemented. It can be decided later what will be the final home for that functionality 👍 But as said, Pyrosm can be a test bed for that (and even home if it looks most feasible option).