conan
conan copied to clipboard
Add check during package download to prevent race condition
Changelog: (Bugfix): In some cases we see race condition when 2 conan instances running at the same time and downloading same package. While locks prevent two conans from downloading and overwriting files in parallel, they currently do not prevent second conan from purging the directory prepared by the first one and re-unzipping files there.
- [ ] Refer to the issue that supports this Pull Request.
- [x] If the issue has missing info, explain the purpose/use case/pain/need that covers this Pull Request.
- [ ] I've read the Contributing guide.
- [ ] I've followed the PEP8 style guides for Python code.
- [ ] I've opened another PR in the Conan docs repo to the
developbranch, documenting this one.
Note: By default this PR will skip the slower tests and will use a limited set of python versions. Check here how to increase the testing level by writing some tags in the current PR body text.
Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.
You have signed the CLA already but the status is still pending? Let us recheck it.
Hi @nanaceba
Thanks for your contribution.
I'd like to understand better:
they currently do not prevent second conan from purging the directory prepared by the first one and re-unzipping files there.
Why is that the current lock is not protecting against this?
As you can see in the broken tests in our CI https://ci.conan.io/blue/organizations/jenkins/ConanTestSuite/detail/PR-12131/4/pipeline, this kind of change is risky and can break things.
Also, the priority right now is 2.0. Concurrency in the 2.0 cache is still not there, but we will eventually work on it, after 2.0. Changes in 1.X branches only happen for regressions and necessary backports and features for the 1.X -> 2.0 migration.
It seems this PR got stale, as the changes didn't pass CI tests, and the changes seemed a bit too risky.
Also, Conan 1.X is not getting this kind of changes anymore, but any significant new changes like this one should target Conan 2.0. In this regard Conan 2.0 has completely removed concurrency for the cache at this moment, and it will be a 2.X roadmap feature to implement it again.
Thanks very much for your contribution!