poetry icon indicating copy to clipboard operation
poetry copied to clipboard

Support for Git Repository Packages which have LFS

Open burnhamd opened this issue 1 year ago • 4 comments

  • [x] I have searched the issues of this repo and believe that this is not a duplicate.
  • [x] I have searched the FAQ and general documentation and believe that my question is not already covered.

Feature Request

Using pip install I am able to use git repositories which have LFS enabled as dependencies in py pyproject.toml. In my projects I often use LFS for storing ML models so that the model and source code are matched in the repo.

I would like to propose allowing git LFS clones when pulling packages from git. My proposal would be to have it as an extra argument in the dependency specification like this: myprivaterepo = { git = "[email protected]:myorganization/myprivaterepo.git", branch = "master", lfs = "enabled"

burnhamd avatar Nov 28 '23 06:11 burnhamd

presumably this just works if you configure poetry to use the system git client (and have configured git correctly)?

if you want "native" support you will need to ask for or contribute it at https://github.com/jelmer/dulwich/

dimbleby avatar Nov 28 '23 08:11 dimbleby

I have bumped into this issue as well.

presumably this just works if you configure poetry to use the system git client (and have configured git correctly)?

As mentioned in this post https://stackoverflow.com/questions/77554348/install-package-from-git-including-git-lfs-files-with-poetry (by the same author of this issue it seems), I did try experimental.system-git-client = true with my system Git configured correctly (Git and LFS installed via Homebrew), but that didn't seem to work.

Oddly enough poetry run pip install git+ssh://[email protected]/org/package.git did work, i.e. LFS was triggered and the files showed up normally not as pointers.

I don't know if there is a way to check that Poetry is indeed calling out to the proper system git client and using the same git config as my shell?

nicolashery avatar Feb 28 '24 14:02 nicolashery

I don't know if there is a way to check that Poetry is indeed calling out to the proper system git client and using the same git config as my shell?

Poetry uses dulwich by default. You can see https://python-poetry.org/docs/dependency-specification/#git-dependencies for instructions on how to use your system git client.

Edit: Missed the part you said you already tried this. Maybe try clearing the cached checkout in your virtual env?

abn avatar Feb 28 '24 15:02 abn