gds_env
gds_env copied to clipboard
Explore using `conda-lock` to generate lockfiles
conda-lock
provides the ability to generate lockfiles for different platforms. This would be a great addition to replace the currently gds_py.yml
as a way to build the same Python environment as the container natively.
An idea of how this could work:
- Build
gds_py
environment on Dockerfile - Export environment into one of the supported input files
- Generate lockfiles for Linux (Arch/arm), macOS (intel/M1), Windows
- Publish lockfiles with each release so the environment is fully recreateable natively on a broader range of machines without conflicts
If the workflow above can work, then we should add CI testing on each architecture.
An alternative to conda-lock
that might work just as well through Github Actions would rely on vanilla conda
ability to build identical environments:
- Build
gds_py
on theDockerfile
- Export environment to
gds_py.yml
as now - Upon commit to this repo, a Github Action would:
- Trigger the build of the environment on a variety of supported architecture/OS/platform options
- Test the environment passes the tests
- Write out a full spec file with:
> conda list --explicit > gds_py-<architecture>-<os>-<platform>.txt
- Commit the spec files to the repository
These spec files would later be able to be used directly to replicate the environment in different machines:
> conda create --name gds --file gds_py-<architecture>-<os>-<platform>.txt
One major drawback of this approach is the end-user needs to know their architecture/OS/platform to select the appropriate install file.
Arch's to be supported (and write explicit files for):
-
win-64
-
osx-64
-
osx-arm64
-
linux-64
-
linux-aarch64