grass
grass copied to clipboard
OSGeo
[Feat] Store extended metadata as JSON in DB
Is your feature request related to a problem? Please describe. Esp. in a TGIS context it would be very useful if extended metadata was stored in the DB so it can be used to retrieve datasets that fulfill specific criteria (think cloud cover for example in satellite data).
Describe the solution you'd like Modern versions of the DBMS backends in GRASS GIS support JSON format:
- SQLite: https://www.sqlite.org/json1.html#:~:text=3.-,Interface%20Overview,a%20sixth%20%22JSON%22%20type.
- PostgreSQL: https://www.postgresql.org/docs/9.3/functions-json.html
That way a variety of metadata content can be stored in one column and the user can still query data not only with regards to time but also other data qualities that can vary between datasets, without extended programming.
Describe alternatives you've considered
Currently, extended metadata is optionally stored in the mapset by selected addons (i.sentinel). Further metadata handling is currently available as addons (wx.metadata) or through r.support. Those could be entry points as well...
Additional context The JSON functionality is fairly new in SQLite and available only as a loadable extension, so this will probably be no default functionality very soon…
Keep in mind – any metadata can be useful also for C modules thus solution should be made in C library with a wrapper for python. See band references for an example when two identical implementations emerge.
Keep in mind – any metadata can be useful also for C modules thus solution should be made in C library with a wrapper for python. See band references for an example when two identical implementations emerge.
Indeed, I think it is comparable to the semantic lable case.
There is a risk of duplicating metadata in both temporal DB and file single file storage (like with semantic lables). One solution could be to store extended metadata as json files (as it is done right now), and provide a function to extract relevant information for a list of maps, e.g. for filtering in TGIS. I cannot contribute with C that much though...
There is also: https://github.com/OSGeo/grass-addons/tree/grass7/src/gui/wxpython/wx.metadata, which is relevant here too...
Another issues to consider is also the number of files (many small files can be challenging on some file systems and in some architectures) as one additional file for thousands of TGIS maps means thousands of additional files...
Including some JSON parser for C as a dependency seems reasonable these days. ...perhaps even YAML.
On the other hand, the good thing about JSON is that it is clear how to access the information, so first Python code can simply write it using the json package and if the need emerges later for C code to read/write the data, JSON C library can be added and used there without a need for an API. This approach may break quickly with a lot of changes to the JSON structure, but may work reasonably well when the code has a good test coverage.
