numba-dpex
numba-dpex copied to clipboard
Reproducible builds and development environment
From time to time we face the situation when code is not changes but CI starts failing. This is usually happens because new development package appears on the channels. We need to control versions for all packages and make transition manually. We should update versions via PRs. If possible we should use automatic updaters like dependabot.
- [ ] Define exact packages in development environment. environment.yml file defines versions but not exact packages.
- [ ] Define exact packages used for build numba-dppy package. Use conda build config.
What to control:
- [ ] packages used for builds
- [ ] environment variables
- [ ] locales
- [ ] configurations in $HOME (i.e.
.condarc
,pip
) - [ ] conda-build package version in build environment https://github.com/IntelPython/numba-dppy/blob/06756c70d9c6951811111731648e8814d7acccbc/.github/workflows/conda-package.yml#L53
- [ ] environment.yml files should be concrete to have versions. Also there should be an easy way to update versions. I.e. create environment
conda env create -f environment.yml
, activate environmentconda activate dev
, update packageconda update numba -c ...
or remove version form environment.yml andconda env update -f environment.yml
orconda search numba -c ...
, then set version explicitly in environment.yml. - [ ] There should be many environment.yml files as they depends on platform (and possibly python version)
Improvements:
- [ ] For creating conda environments for builds and tests use environment file or spec-file.txt placed in
scripts
folder
Tools:
- [ ] Dependabot for automatic updating dependencies and creating PRs for it
- [x] Docker could make simpler control environments
- [ ] https://github.com/conda-incubator/conda-lock could be useful or controlling versions
Examples:
- [x] #582 fails because numba 0.55 is not released and we use dev packages.