OLS icon indicating copy to clipboard operation
OLS copied to clipboard

How to synchronize ontologies between OLS instances?

Open KonradHoeffner opened this issue 3 years ago • 5 comments

We want to setup synchronization between two OLS instances where one instance A is the source and another B is the target. In regular intervals or on a manual triger, ontologies from A should be imported into B if they don't exist yet or if there have been changes. Potentially this could also work in both directions. Does something like this exist or if not could you give me advice on where I could best start implementing this?

KonradHoeffner avatar Apr 25 '22 07:04 KonradHoeffner

Nothing like this exists at the moment. If I were to implement it I would perhaps think about modifying the config importer to import the configuration from the other instance, but I'm not sure how easy this would be.

jamesamcl avatar Apr 25 '22 14:04 jamesamcl

Thanks, I will familiarize myself with the code base and then try to implement the feature at the config importer! However it may take a while as I don't have any prior experience with Spring.

KonradHoeffner avatar May 18 '22 13:05 KonradHoeffner

https://www.ebi.ac.uk/ols/docs/index states:

How often does OLS check for updates in external ontology files? OLS updates nightly to always provide the latest ontology versions

Where can I know more about this update mechanism? Maybe this can be used as a synchronization mechanism? Or does this work outside of OLS by rebuilding and restarting the docker container in a cron job?

KonradHoeffner avatar May 20 '22 09:05 KonradHoeffner

The scheduling happens outside of OLS. However, it is the indexing application that decides what needs to be reloaded.

henrietteharmse avatar May 20 '22 09:05 henrietteharmse

In the end, we found it easier to develop a small stand alone command line application at https://github.com/annosaxfdm/olsync that uses the OLS API to read in the ontology configuration of one OLS instance and that transforms this into an OLS configuration file so that those ontologies can be loaded into another OLS instance. Maybe this is useful for someone else. While it could probably be implemented more directly inside OLS itself, we found it easier to decouple it into a separate component that can be loaded inside a Docker compose file and that can be used alongside the original OLS without the need to necessarily maintain a fork.

KonradHoeffner avatar Nov 17 '22 08:11 KonradHoeffner