prefix-dev
Lockfiles and avoiding re-solving with multi-subpackage recipes
One thing that plagues our build efforts is when the result of a solve changes during the time when a PR is under development. This is most often caused by version bumps in external packages and pinning that allows it. It causes churn and wastes developer time.
What we would like to do is to freeze our dependency lists for each environment created by rattler-build at the first build of a PR, and subsequently use this frozen dependency list for the rest of the iterations in a PR. This is kind of related to https://rattler.build/latest/rebuild/, but the rendered recipe there only has one subpackage of the many in the original recipe.yaml.
Here's an example original recipe: https://github.com/rapidsai/cudf/blob/branch-25.08/conda/recipes/libcudf/recipe.yaml
and a rendered_recipe.yaml is attached (it lacks the other outputs, such as https://github.com/rapidsai/cudf/blob/branch-25.08/conda/recipes/libcudf/recipe.yaml#L149)
Do you have any recommendations for how to get a "locked" recipe that preserves multiple subpackages?
Hey @msarahan - I think there is merit to these ideas and things could certainly be done, one way or another.
An easier solution might be to introduce the timestamp limiting that we can alreay give to rattler, so that we wouldn't consider any conda packages released after TIMESTAMP.
Adding a CLI flag for that would be pretty straight forward. Then you just need to wire up some code to figure out when a given PR was opened :)
https://github.com/prefix-dev/rattler-build/blob/5baeaf1bf3ea58e52c3a7b5ce3ffcb51711f1974/src/render/solver.rs#L103-L108
This is the place for the argument
Great idea! Thanks. That will serve the purpose of avoiding unexpected updates, which is our biggest problem. Lockfiles would still be nice to avoid paying the solving cost, but that's just an optimization more than fixing a bad behavior. I'm going to leave the issue open, but feel free to close if you'd like. I'd consider this a low-priority backlog task.
Here is an implementation: https://github.com/prefix-dev/rattler-build/pull/1759