Tracking Issue for `-Zgit`

[weihanglo]

Summary

Original issue: #1171. See also #11813 Implementation: #13252 Documentation: https://doc.rust-lang.org/nightly/cargo/reference/unstable.html#git

-Zgit is a collection of new Git features in Cargo. Currently we have experiments on

• Shallow clones for git dependencies
• Shallow clones for registry indexes

Unresolved Issues

• [ ] No native support of local file shallow clones for either gitoxide or libgit2
  • https://github.com/Byron/gitoxide/issues/734
  • https://github.com/libgit2/libgit2/issues/6634
• [ ] Add tests for git2 shallow clones. This is blocked on the support of file: protocol.
  • gitoxide also shells out to git CLI when talking to file protocol. We might want to add some container based tests for testing non-file remote protocols for shallow clones.
• [x] libgit2 bug: rust-lang/git2-rs#1084
  • Fixed in libgit2 v1.9.0
• [ ] Is *-shallow directory layout optimal? Can we avoid that but remain compatible?
  • Perhaps switch to bare repos together? #9503
• [ ] net.git-fetch-with-cli supports shallow fetch https://github.com/rust-lang/cargo/issues/14956

Future Extensions

See "Future Extensions" in #11813. Most of them depend on upstream features in libgit2.

About tracking issues

Tracking issues are used to record the overall progress of implementation. They are also used as hubs connecting to other relevant issues, e.g., bugs or open design questions. A tracking issue is however not meant for large scale discussion, questions, or bug reports about a feature. Instead, open a dedicated issue for the specific matter and add the relevant feature gate label.

[NoelJacob]

My ~/.cargo/config.toml looks like this:

git.shallow-index = true
git.shallow-deps = true
gitoxide.fetch = true
gitoxide.checkout = true

But on running cargo help I get :

>  cargo help
error: missing field `internal_use_git2`

>  cargo help
warning: unused config key `unstable.git.shallow-index` in `/home/noel/.cargo/config.toml`
warning: unused config key `unstable.git.shallow-deps` in `/home/noel/.cargo/config.toml`
error: missing field `shallow_index`

Keep repeating cargo <command> and the error seems to repeat between the two. Passing -Zgit -Zgitoxide seems to not give error.

Also modifying it to:

[unstable.git]
shallow_index = true
shallow_deps = true

[unstable.gitoxide]
fetch = true
checkout = true
internal_use_git2 = false

does not give any error but I don't know if its working or not.

[weihanglo]

Under ~/.cargo/git, check if the git clone contains a -shallow suffix.

~/.cargo/git $ tree -L 2
./
├── checkouts/
│  └── empty-library-0161899a0cd3bb43-shallow/
└── db/
   └── empty-library-0161899a0cd3bb43-shallow/

As of this writing, you need this in your .cargo/config.toml:

[unstable.git]
shallow_index = true
shallow_deps = true
[unstable.gitoxide]
fetch = true
checkout = true
list_files = true
internal_use_git2 = false

It is not ergonomic, but while it is still unstable, I am not too concerned. If people want to fix it, go for it! See https://github.com/rust-lang/cargo/issues/11813#issuecomment-1817517629 as well.

[linyihai]

(Maybe a bit off topic) what is the relationship between -Zgit and -Zgitoxide? I see both features being developed at the same time, will they be stable in the future?

[weihanglo]

-Zgitoxide was first developed. The shallow clone support in libgit2 came out after and then we decided to split into -Zgit and -Zgitoxide. The former controls Git related unstable features in general, and with the latter some of those operations will be performed by gitoxide.

[soloturn]

for build environments partial clones in the sense of

--filter=tree:0

make most sense. shallow clones i the sense of -depth=1 may have undesired side effects. see e.g. here for details: https://github.blog/2020-12-21-get-up-to-speed-with-partial-clone-and-shallow-clone/

@weihanglo what you think about ditching "shallow" in general as dead end and replace with "partial"? @cesfahani did a promising start: https://github.com/Byron/gitoxide/issues/1046 .

[weihanglo]

Sparse checkout and partial clone have been discussed in https://github.com/rust-lang/cargo/issues/11165.

It is possible that Cargo ditch or adopt anything for -Zgit since it is still an experimental unstable feature. See Byron's comment https://github.com/rust-lang/cargo/issues/11165#issuecomment-2029220994 for some integration challenges. I can see that download blobs on demand would be helpful for fetching registry index. I am not sure about git dependencies. People might want a "fetch-first-then-offline-build" environment, and we need to model a lazy mechanism for it in Cargo, which is also a bit challenging.