plenary.nvim icon indicating copy to clipboard operation
plenary.nvim copied to clipboard

Consider cleaning up the `.git` folder to reduce the large repo size

Open NTBBloodbath opened this issue 4 years ago • 6 comments

Hey, I've had to clone the repository again, and have found that it's a bit heavy (~12MB) even to download and can cause it to fail on slow connections with the default packer settings (60s timeout).

After doing a little search I have found that it's a nvim.appimage.1 file that doesn't exist and that it's stored in the .pack file.

Here are the steps I took and how to reduce the repository size locally by up to 90% (1.2MB), all steps must be done from the repository root:

  • To see the 10 biggest files:
git verify-pack -v .git/objects/pack/pack-26ca0b79e92d096b8ce4eb32e16238ca74461801.idx \
| sort -k 3 -n \
| tail -10
  • To see what each file is (the important and bigger one is 943f15):
git rev-list --objects --all | grep 'first few chars of the sha1 from previous output'
  • Clean up your git by removing all of those unnecessary files:
git filter-branch --index-filter 'git rm --cached --ignore-unmatch nvim.appimage.1' -- --all
rm -Rf .git/refs/original
rm -Rf .git/logs/
git gc --aggressive --prune=now
  • Verify your size-pack, should be a lot smaller now:
git count-objects -v

NTBBloodbath avatar May 21 '21 23:05 NTBBloodbath

Is there a way I can do this for other people? I just ran these locally but I don't think it did anything for pushing... haha I'm not an expert in this area of git

tjdevries avatar Jun 17 '21 18:06 tjdevries

Yes, after cleaning it should be enough to do git push origin --force -all.

Edit: it will rewrite all the commits because of the git filter-branch command, but I think it's a better approach than creating a new branch to start again and lose the git commits logs.

NTBBloodbath avatar Jun 17 '21 22:06 NTBBloodbath

Just a small data point: the slow download is in fact an issue with Telescope's minimal.lua, since it needs to bootstrap plenary (which it does async, so there's a good chance the user will run into problems if they miss the "ready" prompt...)

clason avatar Aug 13 '21 10:08 clason

Maybe https://github.com/newren/git-filter-repo is a better approach?

  1. pip3 install git-filter-repo
  2. git filter-repo --strip-blobs-bigger-than 1M
  3. git push origin --force

This will still rewrite history, of course, breaking current clones (and PRs), though.

clason avatar Aug 13 '21 10:08 clason

OK, I dug a little deeper. The issue is that packer clones with --no-single-branch, which pulls in metadata for all current branches -- some of which apparently still have the appimage in them. Pruning the offending branches should be enough to reduce the footprint of the clones.

clason avatar Aug 13 '21 11:08 clason

And it's (only) the path branch, to be precise. (Still a good idea to delete the other closed/merged branches as well.)

clason avatar Aug 13 '21 13:08 clason