nextflow-io
[Bug/Behavior] changes to Conda.cacheDir (config) prevents resumability (workflow caching)
Bug/Feature
Expected behavior and actual behavior
Changing Conda.cacheDir in the config causes the workflow cache to miss (resumability to fail). We prefer that nextflow resumes and reuses the existing process-results. Any missing conda environments in the new conda.cacheDir would be recreated, since they are reproducible.
It seems the caching is based on the name of the environment path, but those are a fairly weak indicator for reproducibility.
This could optionally be gated behind the cache = "lenient" directive, but i think its preferable for most users to have caching whenever possible.
Steps to reproduce the problem
Try to resume a pipeline with cached results and conda, but a changed conda.cacheDir in the workflow config from the initial invocation.
workaround
Moving and symlinking the conda cachedir to the new location.
Environment
- Nextflow version: 25.10.0
- Java version: 21.0.9
- Operating system:Linux
- Bash version: 4.4.23
- conda: 25.9.1
Additional context
There are some circumstances like running out of storage or otherwise having to change the conda install, where this can result in the loss of days of computing time. With the rapid growth of conda cacheDirs and limited hpc or cloud filesystems on /home, this can happen fairly quickly.
