geojson skip null property values

[andrewharvey]

Any chance for a GeoJSON format layer creation option to skip outputting null properties?

So outputting something like:

..."properties": { a: 1, d: 2, f: 1 }.... ..."properties": { b: 1, c: 2, d: 2 }...

instead of

..."properties": { a: 1, b: null, c: null, d: 2, e: null, f: 1 }... ..."properties": { a: null, b: 1, c: 2, d: 2, e: null, f: null }...

ref: https://gis.stackexchange.com/questions/132231/remove-null-attributes-from-geojson

[rouault]

You are probably using a too old GDAL version. Since GDAL 2.2.0, the concept of empty field values has been added and will be correctly used in that case.

Given in.json

{
"type": "FeatureCollection",
"features": [
{ "type": "Feature", "properties": { "a": 1}, "geometry": null },
{ "type": "Feature", "properties": { "b": 2 }, "geometry": null }
]
}

$ ogr2ogr -f geojson /vsistdout/ in.json -t_srs EPSG:3857 generates

{
"type": "FeatureCollection",
"name": "in",
"crs": { "type": "name", "properties": { "name": "urn:ogc:def:crs:EPSG::3857" } },
"features": [
{ "type": "Feature", "properties": { "a": 1 }, "geometry": null },
{ "type": "Feature", "properties": { "b": 2 }, "geometry": null }
]
}

[andrewharvey]

Ah, so it's a separate issue then because your example works fine for me GDAL 2.3.1. I should have explained better.

When I use ogr2ogr to convert PostGIS data with some null values to GeoJSON, GeoJSON outputs hard null values instead of just dropping the attribute, which is a different issue to what I described here.

[jratike80]

If you import such GeoJSON back to PostGIS it would change the schema by dropping the all-null attributes which may feel not correct for some users. I think that users should also have an option for writing schema-preserving GeoJSON that roundtrips nicely.

[rouault]

When I use ogr2ogr to convert PostGIS data with some null values to GeoJSON, GeoJSON outputs hard null values instead of just dropping the attribute, which is a different issue to what I described here.

This is expected . In a SQL database, there is no concept of empty/missing field. There is just NULL. Yeah, maybe a -null_as_unset option to ogr2ogr could make sense.

[Blake-Loveland]

I have to post process these now to remove the key,value pairs when the value is null. GIS datasets often have a lot of fields, and little data which leads to huge files.

Since JSON easily supports varied attributes on an object, I think it makes sense to enhance the format export, with the option to take advantage of a feature of the format being exported, in this case json.

I can do the simple scripting to remove them after the export, but I do not think I could figure out where/how to make the change in the GDAL codebase, so unfortunately, I can not make a PR to address it.

Cheers, thank you to everyone who contributes to OSGeo!

[rduivenvoorde]

As an extra argument: at geojson.io it is possible to use some (svg like) styling (see https://github.com/mapbox/simplestyle-spec/tree/master/1.1.0) defined as 'normal properties'.

So IF I wanted to create (partly) styled geojson, I should create (for example a) geopackage with all style props (at least 8 looking at simplestyle spec). And only using fill for example. Now creating geojson with it I will end up with all props with null values, so yes a null_as_unset option would be cool I think :-)

Though now I'm wondering how gdal is handling such geojson when reading? Adding attributes on the go during read?

[tordans]

+1 for this and here is my use case:

We have OSM Data which we put into jsonb column using osm2pgsql (it's just easier than having separate columns). We now need an export of the data, which does not play nice with jsonb. Instead we select the distinct keys from the jsonb and write a dynamic sql query where we "flatten" the jsonb. That all works OK, but it does result is a lot of foo: NULL data being added. Here is some code.

Interestingly when I change the format to fgb the null values do not show up when I open the file with https://play.placemark.io/ afterwards.

[verhovsky]

I'm also working with OpenStreetMap data and this issue makes my data take 3 GB instead of 121 MB because the data is very diverse and if one building in a city uses an uncommon tag like sauna=yes, that means all 400,000 buildings will have "sauna":null,

import osmnx as ox

bbox = [-114.3387482,   50.8341488, -113.8194342,   51.2270627]
gdf = ox.features.features_from_bbox(bbox, {"building": True})
gdf.to_file("calgary_buildings.geojson", driver="GeoJSON")