uv doesn't correctly checkout Git dependencies with Git LFS assets

[sydduckworth]

I'm working on a project that depends on a private Git repository where some of the project's assets are stored in Git LFS (in this case Pytorch models).

My project uses Rye as its package manager. When configured to use Pip, everything works as expected. When configured to use uv, the LFS objects aren't checked out correctly.

I assume a blocker for uv supporting LFS is the fact that the git2 crate also doesn't support LFS (rust-lang/git2-rs#956) and there's no indication support will be added in the future.

Platform: Ubuntu 22.04 uv version: 0.1.39

[vfilter]

Same issue here. Makes uv unfortunately unusable for us since we have lfs in a private github repo.

[zanieb]

We actually stopped using the git2 crate so we may be able to support this easier now? cc @ibraheemdev

[gusutabopb]

Still running into this issue using uv 0.3.0.

uv fails due to git lfs failing due to "smudge errors".

Sample command:

uv pip install git+https://[email protected]/repo.git

pip works fine, so my current workaround is:

uv pip install pip
uv run pip install git+https://[email protected]/repo.git

Cloning the repo locally and then installing from a local path works fine too.

[vfilter]

Adding another workaround here. Running git lfs install --force --skip-smudge before installing the dependency from your private repo did solve it for us. I was able to install with uv add after that.

[falckt]

Just for reference, I tried setting the respective settings @vfilter suggested in https://github.com/astral-sh/uv/issues/3312#issuecomment-2323443935 as environment variables

$ env | grep -e ^GIT_CONFIG
GIT_CONFIG_KEY_0=filter.lfs.smudge
GIT_CONFIG_VALUE_0=git-lfs smudge --skip -- %f
GIT_CONFIG_COUNT=1

which is also picked up by git

$ git config --get filter.lfs.smudge         
git-lfs smudge --skip -- %f

However when running uv sync --upgrade-package <my-package-in-git> I still get the smudge filter error.

@zanieb could it be that uv does not pass on the environment variables to the underlying git process? (Setting them via the config file works).

[falckt]

No clue how, but setting the environment variable GIT_LFS_SKIP_SMUDGE=1 does have the desired effect.

This old comment suggests that certain local checkouts might trigger these problems https://github.com/git-lfs/git-lfs/issues/2518#issuecomment-863218693. Not sure whether that is the case here though.

[xela-95]

Hi! This issue is blocking me from adopting pixi, is there any news on this?

[alan-cooney-dsit]

No clue how, but setting the environment variable GIT_LFS_SKIP_SMUDGE=1 does have the desired effect.

This just turns off getting LFS assets btw.

Also very keen for this feature

[zanieb]

Someone want to share a minimal reproduction for this?

[grebnetiew]

Sure.

$ uv pip install git+https://github.com/grebnetiew/lfs-py.git
Updating https://github.com/grebnetiew/lfs-py.git (HEAD)
error: Git operation failed
  Caused by: process didn't exit successfully: `git reset --hard 1abf996d38d769939d2a4f5e06d1afc8e8a91817` (exit status: 128)
--- stderr
Downloading datafile.bin (39 B)
Error downloading object: datafile.bin (4502c72): Smudge error: Error downloading datafile.bin (4502c722f9bcf5f68951d9b41b74c2de404de55cea347c92b0a7d608cd3d9c88): error transferring "4502c722f9bcf5f68951d9b41b74c2de404de55cea347c92b0a7d608cd3d9c88": [0] remote missing object 4502c722f9bcf5f68951d9b41b74c2de404de55cea347c92b0a7d608cd3d9c88

Errors logged to '/home/erieke/.cache/uv/git-v0/checkouts/634788a817b48cd6/1abf996/.git/lfs/logs/20241006T194309.74056189.log'.
Use `git lfs logs last` to view the log.
error: external filter 'git-lfs filter-process' failed
fatal: datafile.bin: smudge filter lfs failed

The workaround above works, though it might not actually get the lfs-file I checked in (I didn't check). Packages that use LFS likely need their LFS files ;)

$ GIT_LFS_SKIP_SMUDGE=1 uv pip install git+https://github.com/grebnetiew/lfs-py.git
 Updated https://github.com/grebnetiew/lfs-py.git (1abf996)
Resolved 1 package in 2.63s
   Built lfs-py @ git+https://github.com/grebnetiew/lfs-py.git@1abf996d38d769939d2a4f5e06d1afc8e8a91817
Prepared 1 package in 643ms
Installed 1 package in 0.86ms
 + lfs-py==0.1.0 (from git+https://github.com/grebnetiew/lfs-py.git@1abf996d38d769939d2a4f5e06d1afc8e8a91817)

[sydduckworth]

Another workaround option if you need LFS for other repos and don't want to manually set environment variables:

Create a new file ~/.gitconfig-nolfs.inc:

[filter "lfs"]
    clean = git-lfs clean -- %f
    smudge = git-lfs smudge --skip -- %f
    process = git-lfs filter-process --skip
    required = true

Create or modify ~/.gitconfig:

[includeIf "gitdir:~/.cache/uv/**"]
    path = ~/.gitconfig-nolfs.inc

This updates your Git config to skip LFS just for uv dependencies.