cylc-flow
cylc-flow copied to clipboard
Buit-in container support
Some users have suggested Cylc needs this capability built in (of course you could roll your own with task scripting, but many users don't know how to do it):
This task should pull down this container, and run this application in it.
(This sort of thing is becoming quite important in the wider world, of course, and even on HPC beyond atmospheric science).
Or more generally built-in environment support (Conda, Docker, Apptainer, modules, etc) as well as integration with cloud systems.
Here's an example I put together for building Conda environments. It would be good to have something more automated. A couple of considerations...
Pull or build:
- Pull down pre-built environments (works for some things, not for others).
- Build environments from recipe files (most flexible, may require internet).
Local or remote:
- Pull/build environments in-situ (may require internet access, not always possible).
- Pull/build environments in one place and transfer where they are needed (e.g.
conda pack
).