open-semantic-search icon indicating copy to clipboard operation
open-semantic-search copied to clipboard

Provide base open-semantic-search containers on docker hub

Open sdinten opened this issue 2 years ago • 10 comments

As @wsldankers stated in #389 we're working to get the docker-compose based version up and running and share the fixes/workarounds through pull requests.

I'd like to propose an enhancement to, as a next step, publish pre-build container images on a public container registry that can be used from the provided docker-compose.yml in this repository.

In our setup we have a gitlab pipeline that builds and publishes these images to the internal container registry. I'd be happy to contribute to this effort, please let me know what you think?

sdinten avatar Dec 16 '21 12:12 sdinten

Had no time to do work on this things last year, but plan to work on this issues (help is very welcome and many thanks to @wsldankers et all for all the work) and after such massive improvement delete the server VM variant https://github.com/opensemanticsearch/open-semantic-search-appliance) which was hard to update.

Mandalka avatar Dec 17 '21 18:12 Mandalka

Todo: Somehow (needs Solr (core of entity api) container up, etl config volume and app container up, so not possible in Dockerfile) or manually once after installation by open semantic search docker user documentation / button in UI we need to add following call (in deb package in src/open-semantic-search-apps/DEBIAN/postinst) to Docker environment to export the initial entities to entities index in Solr:

# setup facets/classes in Open Semantic ETL facet config and entities in Open Semantic Entity Search API
python3 /var/lib/opensemanticsearch/manage.py entities

Maybe at start of the app container if not yet done / missing config or if certain configfile missing this python call, so no manual step for users.

Maybe if empty /etc/opensemanticsearch/facets as indicator? Hm, won't work on docker image updates, but maybe on updates a manual step is ok.

Will integrate such kind of initialization with an docker entry script.

Mandalka avatar Dec 27 '21 10:12 Mandalka

Todo: Share volume with custom OCR dictionary created by thesaurus and ontologies to Tika container.

Mandalka avatar Dec 27 '21 10:12 Mandalka

I'll add a version info into version / status config file and prove on entry of app container, if some upgrade-calls have to be made (and do so if yes) before start of apache2 as generic approach for future (automatic) updates.

Mandalka avatar Dec 27 '21 10:12 Mandalka

Todo: Move autocommit settings to core config since Solr docker image doesn't use solr.in.sh: https://github.com/opensemanticsearch/solr.deb/issues/17

Mandalka avatar Dec 27 '21 11:12 Mandalka

Initialization of things which need multiple containers running now done automatically in docker-entrypoint.sh

Mandalka avatar Dec 28 '21 09:12 Mandalka

Solr should be preconfigured to more max RAM than default (only -Xmx512m)

Mandalka avatar Dec 28 '21 11:12 Mandalka

Todo: documentation how to set django admin pw in docker container

Mandalka avatar Dec 28 '21 16:12 Mandalka

Integrated task queue management and monitoring ui Celery Flower into the docker environment by #323 / commit https://github.com/opensemanticsearch/open-semantic-search/commit/160ff7224039c4586b013a760d5843ec06369c61 (user access by reverse proxy on ui / apps container, so no additional ports/hostnames to manage)

Mandalka avatar Feb 20 '22 08:02 Mandalka

Added (over)write of config files with settings from web admin ui (django db) to entrypoint.

Must this be conditional (do not overwrite settings in config files with settings from web admin ui) for someone?

Mandalka avatar Feb 28 '22 11:02 Mandalka