sapling icon indicating copy to clipboard operation
sapling copied to clipboard

Git LFS Support

Open alex-statsig opened this issue 2 years ago • 9 comments
trafficstars

https://sapling-scm.com/docs/scale/axes#size-of-files talks about how sapling "speaks the git lfs protocol", but I can't find any documentation on how to get this working. I have an existing repository that uses git lfs for images, and can't get sapling to resolve those LFS pointers.

A few other articles also seem to indicate it should work fine: https://engineering.fb.com/2022/11/15/open-source/sapling-source-control-scalable/

However this issue also made me skeptical: https://github.com/facebook/sapling/issues/182

alex-statsig avatar May 25 '23 01:05 alex-statsig

Hi @alex-statsig. Currently there is no actual support for using Git LFS with actual Git repos. The Git-LFS-like support we mention in our docs and in the engineering post refers to how we handle things internally.

Having actual Git LFS support for Git repos is something in our to-do list, however

sggutier avatar May 25 '23 22:05 sggutier

Hi @alex-statsig. Currently there is no actual support for using Git LFS with actual Git repos. The Git-LFS-like support we mention in our docs and in the engineering post refers to how we handle things internally.

Having actual Git LFS support for Git repos is something in our to-do list, however

Thanks for the update. Our company has migrated over from Git and things have mostly been smooth, but this is one of the small blockers for some teams. If there is any rough ETA that can be shared, that would be greatly appreciated!

alex-statsig avatar May 31 '23 04:05 alex-statsig

Update: Sorry to disappoint but after discussion with internal teams, we decided to not implement Git LFS support in Sapling, at least in this year.

While the Git LFS might sound simple at first sight, we also need to consider watchman and edenfs compatibility, which makes it more complicated. Note the upstream Mercurial community also considered related features "unloved" or "last resort".

We used to have an internal team that wants this Git LFS support so we put this as part of our plan. After in-depth discussions, we found their use-case can be achieved without (complicated) support from source control. The team uses Git LFS to store large blobs to avoid overloading the Git servers. The large blobs are used during building. We suggested changing the build logic to download the large blobs instead, then store the large blobs outside (ex. .gitignored) source control, optionally with some hooks to fetch the blobs if they don't want to regress build time. We think this is overall simpler, and they took the suggestion.

quark-zju avatar Aug 15 '23 20:08 quark-zju

Thanks for the information. We'll look into a similar alternative, but I think that setup will be fairly involved.

We also use it to prevent storing large blobs, in particular of static images (or some PDFs) that are stored all around our monorepo. Most importantly I think, we don't want to download them when running CI on every github workflow on every version of PRs. We only want to download them when deploying builds, or for local development. We use actions/checkout@v3 and can set lfs: false/true as needed for the workflow.

alex-statsig avatar Aug 16 '23 16:08 alex-statsig

@quark-zju Unfortunately Github has pushed LFS for our GHE instance for scaling. Changing the workflow to introduce sapling is a non-starter so this means essentially Sapling isn't usable in our environment.

Also LFS isn't just used for building. It's for any large file in a repo. So I am not sure your workaround fits all (or even the majority) of LFS usecases.

nemith avatar Sep 05 '23 19:09 nemith

+1 for LFS support. Definitely a point of friction in adopting sapling. Any thoughts on a sketchy work around for it? I may try to magic up the suggested solution above in a rust/build.rs way to autodownload the files if they're not there but it really breaks the hermetic properties of the build :-/ FYI: @gregor-netdebug.

capveg-netdebug avatar Sep 07 '23 17:09 capveg-netdebug

I'm trying to work around missing lfs support from sapling. Naively, I tried to manually copy-paste the files from a git copy of the repo, but this marks the files as "modified" within Sapling. Is it possible to ignore further changes to (lfs) files without adding them to .gitignore?

TiddoLangerak avatar Oct 24 '23 07:10 TiddoLangerak

I'm trying to work around missing lfs support from sapling. Naively, I tried to manually copy-paste the files from a git copy of the repo, but this marks the files as "modified" within Sapling. Is it possible to ignore further changes to (lfs) files without adding them to .gitignore?

FYI: I 'worked around' this by taking the small third-party library binary out of LFS and just checked in normally. Not a great work around but the library never updates so it seems to be working for us. If anyone finds a better work around, I'd love to hear it.

capveg-netdebug avatar Oct 24 '23 16:10 capveg-netdebug

FYI: I 'worked around' this by taking the small third-party library binary out of LFS and just checked in normally. Not a great work around but the library never updates so it seems to be working for us. If anyone finds a better work around, I'd love to hear it.

Unfortunately, this only works if the files are small, and if adopting sapling with the full team. Neither holds for me...

TiddoLangerak avatar Oct 25 '23 15:10 TiddoLangerak