code-review
code-review copied to clipboard
Add Taskcluster cache for local Mercurial clones
In order to actually use the work done in https://github.com/mozilla/code-review/pull/1517, we need to enable local cloning. In order to enable local cloning, we need to make sure we don't clone from scratch in each task, but we rely on a Taskcluster cache.
I'm looking into this and have a few questions:
- there already exists firefox source code checkouts in Taskcluster cache, but they are restricted to tasks ran by run-task: would you consider moving the code-review bot execution to run-task ?
- if using custom caches for the code review bot, we would need one cache per repository.
- when the cache is empty, some code-review bot tasks would need to populate it (initial clone), and will take longer: is that ok ?
I noticed that we already have cache enabled for the Integration hook.
Unfortunately the hook on Community TC does not successfully run anymore due to a missing secret.
We could restore that functionality, and test the clone & re-use from cache functionalities from that hook before updating runtime tasks
WDYT @marco-c ?
I spent a bit of time today on this topic and noticed a few things:
- the integration test is broken because of invalid Taskcluster proxy url and automated conversion from docker-worker to generic-worker. As I do not know what's the long term plan regarding this hook & its viability, I stopped looking into it.
- I made a first PR to control directly the mercurial cache behaviour from the CLI args, removing Taskcluster "production" check, and simplifying the overall code. In the end, the taskcluster cache is usable because I removed the tempdir path creation. As
robust_checkoutis already used, the cache will be automatically purged & setup at the correct version on each run - The mercurial version tools extension is not at the latest version (but just lack 1 whitespace changeset)
- The mercurial version is outdated (6.4.5 in code vs 6.7.4 on pypi)
- I have issues running locally related to sentry-sdk (probably an invalid version locally that has changed some modules)
Next steps:
- get a review from @Archaeopteryx or @marco-c
- enable cache on https://hg.mozilla.org/ci/ci-configuration/file/tip/hooks/project-relman/code-review-testing.yml
I'm looking into this and have a few questions:
* there already exists [firefox source code checkouts](https://firefox-source-docs.mozilla.org/taskcluster/caches.html#version-control-caches) in Taskcluster cache, but they are restricted to tasks ran by run-task: would you consider moving the code-review bot execution to run-task ?
Yes, but I'm not sure what would be needed to do that. Long term, we might even consider moving the review bot into mozilla-central itself, which would reduce the maintenance a lot.
* if using custom caches for the code review bot, we would need one cache per repository. * when the cache is empty, some code-review bot tasks would need to populate it (initial clone), and will take longer: is that ok ?
Yeah I guess there's not way around that.
Regarding the integration hook, I think it has been broken for a while. It would be useful to have it working again, though it's not a huge priority.