cargo icon indicating copy to clipboard operation
cargo copied to clipboard

Cache submodule into git db

Open avnyu opened this issue 2 months ago • 6 comments

What does this PR try to resolve?

My attempt to continue https://github.com/rust-lang/cargo/pull/10279.

This:

  • Moves the code for creating git db into GitSource::update_db
  • Creates GitSource for each submodules
  • Replaces fetch inside update_submodule by GitSource::update_db and db.copy_to
  • Removes recursive update_submodules calls cos db.copy_to already recursive.

Fixes https://github.com/rust-lang/cargo/issues/7987.

How to test and review this PR?

I tested using the original pull method:

~/.cargo/target/debug/cargo update -p boring --precise 46787b7b6909cadf81cf3a8cd9dc351c9efdfdbd
~/.cargo/target/debug/cargo update -p boring --precise c037a438f8d7b91533524570237afcfeffffe496

and confirmed that the time to do the second update is negligible. Also test if it can fetch submodule offline using the downloaded git db

git clone https://github.com/pop-os/cosmic-files && cd cosmic-files
~/.cargo/target/debug/cargo fetch --locked --target=$(rustc --print host-tuple)
rm -rf ~/.cargo/git/checkout ~/.cargo/target/debug/cargo fetch --locked --target=$(rustc --print host-tuple) --offline

avnyu avatar Nov 12 '25 16:11 avnyu

r? @weihanglo

rustbot has assigned @weihanglo. They will have a look at your PR within the next two weeks and either review your PR or reassign to another reviewer.

Use r? to explicitly pick a reviewer

rustbot avatar Nov 12 '25 16:11 rustbot

It would help reviewers if you break this down into atomic commits. For example, by making a commit for extracting update_db, it becomes clear whether it was purely an extraction or there are also other changes.

See https://doc.crates.io/contrib/process/working-on-cargo.html#submitting-a-pull-request

epage avatar Nov 12 '25 16:11 epage

This PR was rebased onto a different master commit. Here's a range-diff highlighting what actually changed.

Rebasing is a normal part of keeping PRs up to date, so no action is needed—this note is just to help reviewers.

rustbot avatar Nov 12 '25 16:11 rustbot

This PR was rebased onto a different master commit. Here's a range-diff highlighting what actually changed.

Rebasing is a normal part of keeping PRs up to date, so no action is needed—this note is just to help reviewers.

rustbot avatar Nov 12 '25 17:11 rustbot

Done, the "extract code into update_db" part is a single commit now. I think the remains part needs to be in the same commit

avnyu avatar Nov 12 '25 17:11 avnyu

This PR was rebased onto a different master commit. Here's a range-diff highlighting what actually changed.

Rebasing is a normal part of keeping PRs up to date, so no action is needed—this note is just to help reviewers.

rustbot avatar Nov 27 '25 08:11 rustbot

@rustbot author

weihanglo avatar Dec 09 '25 14:12 weihanglo

Reminder, once the PR becomes ready for a review, use @rustbot ready.

rustbot avatar Dec 09 '25 14:12 rustbot

This PR was rebased onto a different master commit. Here's a range-diff highlighting what actually changed.

Rebasing is a normal part of keeping PRs up to date, so no action is needed—this note is just to help reviewers.

rustbot avatar Dec 10 '25 03:12 rustbot

Sorry for the late response, I force-pushed the commits. About new tests, if I am not mistaking, it should be only about the git db cache, not the correctness of git repo checkouts, as it should be already covered? Currently I can't find api for manipulating the cached git db, or example about testing the git cache db

avnyu avatar Dec 10 '25 04:12 avnyu

Overall looks like. Let me know if you plan to reorganize the commits, or I can do that for you. Thanks!

View changes since this review

If you can do it, please do. Thanks!

avnyu avatar Dec 13 '25 12:12 avnyu