geojson-to-sqlite
geojson-to-sqlite copied to clipboard
`--spatialite` fails if features have no properties
https://raw.githubusercontent.com/datanews/amtrak-geojson/master/amtrak-combined.geojson
Has a feature like this:
"properties": { }
This fails with an error if you use the --spatialite option:
geojson-to-sqlite /tmp/amtrak.db amtrak /tmp/amtrak-combined.geojson --spatialite
Traceback (most recent call last):
File "/Users/simon/.local/bin/geojson-to-sqlite", line 8, in <module>
sys.exit(cli())
File "/Users/simon/.local/pipx/venvs/geojson-to-sqlite/lib/python3.9/site-packages/click/core.py", line 1134, in __call__
return self.main(*args, **kwargs)
File "/Users/simon/.local/pipx/venvs/geojson-to-sqlite/lib/python3.9/site-packages/click/core.py", line 1059, in main
rv = self.invoke(ctx)
File "/Users/simon/.local/pipx/venvs/geojson-to-sqlite/lib/python3.9/site-packages/click/core.py", line 1401, in invoke
return ctx.invoke(self.callback, **ctx.params)
File "/Users/simon/.local/pipx/venvs/geojson-to-sqlite/lib/python3.9/site-packages/click/core.py", line 767, in invoke
return __callback(*args, **kwargs)
File "/Users/simon/.local/pipx/venvs/geojson-to-sqlite/lib/python3.9/site-packages/geojson_to_sqlite/cli.py", line 32, in cli
utils.import_features(
File "/Users/simon/.local/pipx/venvs/geojson-to-sqlite/lib/python3.9/site-packages/geojson_to_sqlite/utils.py", line 69, in import_features
db[table].create(column_types, pk=pk)
File "/Users/simon/.local/pipx/venvs/geojson-to-sqlite/lib/python3.9/site-packages/sqlite_utils/db.py", line 863, in create
self.db.create_table(
File "/Users/simon/.local/pipx/venvs/geojson-to-sqlite/lib/python3.9/site-packages/sqlite_utils/db.py", line 517, in create_table
self.execute(sql)
File "/Users/simon/.local/pipx/venvs/geojson-to-sqlite/lib/python3.9/site-packages/sqlite_utils/db.py", line 236, in execute
return self.conn.execute(sql)
sqlite3.OperationalError: near ")": syntax error
Here's why:
https://github.com/simonw/geojson-to-sqlite/blob/17c2d0995bec989fa38ef3a593649908833b7b0a/geojson_to_sqlite/utils.py#L70-L75
Since geometry is added later the call to db[table].create(column_types, pk=pk) has an empty column_types dictionary.
Really hard to know what to do about this one - the GeoJSON object here really is just a geometry with nothing else. There's not even an ID that can be used as a primary key.
So we really want to create a SQLite table with just a rowid. Is that even possible?
https://stackoverflow.com/a/36926664/6083 suggests using this:
CREATE TABLE tablename (rowid INTEGER PRIMARY KEY) WITHOUT ROWID;
That makes me a little bit nervous, see https://www.sqlite.org/withoutrowid.html - it's not clear to me if this kind of "fake" rowid table will cause weird problems in the future, especially if I start using tricks like this one: https://til.simonwillison.net/sqlite/track-timestamped-changes-to-a-table
I'm tempted to create a id column that is primary key integer and hence will auto-increment.
I think the safest thing to do here would be to create the table with a _temp column, add the geometry column and then drop that _temp column.
Fixed it! Here's that GeoJSON imported and rendered using https://datasette.io/plugins/datasette-geojson-map
I think I spotted a bug: https://github.com/simonw/geojson-to-sqlite/blob/23bdb74f857b067fbddbc924fd27b64d3441a46c/geojson_to_sqlite/utils.py#L70-L77
remove_tmp_column is never switched to True.