cvat-opencv
cvat-opencv copied to clipboard
Pin Python dependencies with pip-tools
Motivation and context
pip dependency resolver was taking extremely long due to complex version resolution prior to this change (>6 hours on CI). This commit introduces version pinning with pip-tools. A wrapper script to update the pinned versions when dependencies are added, or when upgrades are needed, is provided.
Fixes #4618
How has this been tested?
CI-nightly build phase seems to pass the error described in #4618. However, the full build still doesn't complete (>6h passed).
Checklist
- [x] I submit my changes into the
developbranch - [ ] I have added a description of my changes into CHANGELOG file
- [ ] I have updated the documentation accordingly
- [ ] I have added tests to cover my changes
- [x] I have linked related issues (read github docs)
- [x] I have increased versions of npm packages if it is necessary (cvat-canvas, cvat-core, cvat-data and cvat-ui)
License
- [x] I submit my code changes under the same MIT License that covers the project. Feel free to contact the maintainers if that's a concern.
- [ ] I have updated the license header for each file (see an example below)
# Copyright (C) 2022 Intel Corporation
#
# SPDX-License-Identifier: MIT
@ngaloppo, thank you for the PR! It seems that the issue linked was with a specific package, and it is resolved now. I'm not sure that we want this specific solution for the develop branch, but it can be a good way to make tagged releases and custom deployments more reproducible and stable against non-fixed dependency updates. @azhavoro, what do you think? Can we run such a script before releases?
Please also check and comment on https://github.com/opencv/cvat/issues/4270.
I quite like pip-compile, and I would support integrating it. That said, I think a few tweaks need to be made to the current approach:
-
We should not immediately drop all version specifiers for the
requirements.infiles. Rather, the initial.infiles should just be renamed from the existing.txtfiles. This will help avoid unexpected changes when people regenerate the.txtfiles later. Later on, we can unpin/relax versions on a case-by-case basis. -
We should use
pip-compile-multito avoid duplication of requirements between the various.txtfiles.pip-compile-multialso automatically finds the files to process, so it'll let us drop the automation script.