gbif
Add MongoDB connector
FWIW, KE-EMu is in the midst of migrating its backend to MongoDB. Other backends will be options, but these too will be stored as JSON documents.
Thanks @dshorthouse. There are currently 22 other open enhancements and everybody is warmly invited to thumbs up their favorites or comment as you did in order to raise their priority in getting attended to.
Sadly, it doesn't look like there's a free (& usable) jdbc connector.
- https://github.com/erh/mongo-jdbc (no development in 8 years)
- http://www.unityjdbc.com/mongojdbc/mongo_jdbc.php ($500 US for client, unknown for server)
- https://www.dbschema.com/mongodb-jdbc-driver.html (unknown)
It does not have to be JDBC, we also added support for Excel sheets through a custom connector. But I would hope we can get away with the same kind of mapping configurations, otherwise the coding would become a little difficult.
Currently the IPT works by configuring multiple sources that need to expose flat rows with columns. These can then be mapped to the terms and core/extension rowTypes.
As long as we can expose MongoDB in, potentially multiple, flat tables that can be accessed by a column index number we are good.
getting a list of documents is probably fine, but how can we flatten an arbitrary nested JSON document? Could a source define a list of JSONPath expressions that mongo-java-driver can evaluate and we can then map terms to as the list of json paths is flat and indexable just as row column numbers?
Would you expose all documents or query for a subset?
Looks like we could use a mongodb projection: https://docs.mongodb.com/manual/tutorial/project-fields-from-query-results/#projection
http://mongodb.github.io/mongo-java-driver/3.5/driver/tutorials/perform-read-operations/#projections