invenio
invenio copied to clipboard
inveniosoftware
Addition of requirements.txt
Hello,
in order to make sure that the installation instructions work regardless of changes in the dependencies, I propose that a requirements.txt file is added. It would freeze the versions used and avoid issues similar to #3737
I fully agree we need a way to install Invenio that doesn't break every other week because a new version of a dependency was released. Especially for new users who just want to try out Invenio this is important.
Today, services are usually providing the requirements.txt file - see e.g. Zenodo. What makes it hard for Invenio is that have many "extra_requires" so one requirements.txt file likely won't do, and figuring out what precisely to put in the requirements is also not obvious due to Python versions, DB versions, ES version etc. Afterwards comes the pain of keeping this file updated.
I'd like to see Invenio becomes very easy to install and get started with, and part of that I think is providing different Invenio flavours (e.g. media archive, ILS, research data) that will actually be much like a service. I think that most likely the requirements.txt files can be added to those Invenio flavours, and we can have a say monthly patch cycle where the requirements.txt gets updated and released.
@lnielsen Maybe we could add one just for the provided Dockerfile? That way the instructions on inveniosoftware.org would work. We could exclude the file from the pypi release.
The Dockerfile is only for demonstration right?
Yes, instructions on inveniosoftware.org must work (and will hopefully work by IUGW :) )
There's two new repos: https://github.com/inveniosoftware/invenio-app https://github.com/inveniosoftware/invenio-app-ils
With the latter being an example of an ILS flavour of Invenio. This flavour would be installed from invenio via something like:
$ pip install invenio[ils]
See feature branch in Invenio for how that would look.
I think requirements.txt can be added to invenio-app-ils and updated every other moths, but still things like postgres vs mysql, es version will be hard to add in, but at least for the other dependencies.
I don't think pinning the PyPI requirements is enough. It happened in the past several times that things broke when something was updated from v1.2.3 to v1.2.4, and if this something isn't a Python package, the pinning won't help to catch this.
Besides, we want to get security updates that may be coming with Flask-Foo v4.5.6 to v4.5.7 upgrade. So it would be good not to "overpin" things too much. We could leave some liberties to account for these cases, but we also had observed some breakage for such patchlevel updates in the past too, since not everybody uses semantic versioning strictly.
Hence, rather than pinning, I think the best would be to simply revive our good old nightly builds and test Docker, Ubuntu14, CentOS7 installation techniques every night, as we used to do with Jenkins on legacy code base. This will make sure that we have latest-greatest demo site features and that the installation procedures always work too.
I can take care of this...
(P.S.1. Note that the demo site was meant to be an "integration testing space" for various Invenio components and features, so pinning packages wouldn't be too good from this point of view either. Rather use latest-greatest packages and catch troubles nightly in a true CI style.)
(P.S.2. Note also that helper scripts are used in both Docker and non-Docker installation technique, so that we are fully sure that both installation methods -- native and docker -- are always OK and that one of them has not fallen apart due to being unused, as was sometimes the case in service overlays.)
@tiborsimko thank you for the clarification. I thought that the docker was only for demonstration, or a starting point for whoever created a new deployment. I didn't think that it was also used as integration test.
For now every module update impacts the getting started on http://inveniosoftware.org/gettingstarted? but the docker deployment should break rarely once all modules are stable.
👍 for nightly builds