Reduce source code size (git repo 1.5G) / source code v1.6.0.tar.gz 434MB

[shawn111]

It need a lot of space to clone goose, do you consider reduce the source code size? It might be painful to filter those files out of repo.

$ git count-objects -vH 
count: 0
size: 0 bytes
in-pack: 685259
packs: 1
size-pack: 1.42 GiB
prune-packable: 0
garbage: 0
size-garbage: 0 bytes

git rev-list --all --objects | git cat-file --batch-check='%(objectname) %(objecttype) %(objectsize) %(rest)' | sort -k3n | tail -n 10

691c8861dd2e1e788bed999d1452053b7b53f79e blob 31370992 crates/goose-ffi/examples/libgoose_ffi.dylib
2a27b3fbdef2c38a880ef5756df1db0bcb3de555 blob 32508440 ui/desktop/src/bin/goosed
446855ae0f90b60972cff151cf10813a02a43270 blob 32643896 ui/desktop/src/bin/goosed
c22131491507d58ef0d43a09660e0d7d25f789ad blob 32643896 ui/desktop/src/bin/goosed
47ecae731e6382ee32d880c149eb25b0f9bf1cb9 blob 33537272 ui/desktop/src/bin/goosed
f6cbad21bfa4da8cf8a847eba3d2aa13edfd5eed blob 33629224 ui/desktop/src/bin/goosed
729bb7b38a29a2384d13352cb62710d03a66baf1 blob 33908154 documentation/prompt-library/node_modules/.cache/webpack/client-development-en/0.pack
dbfa26d83c96640b1ed844e0f5d720d48c42d9ab blob 47275520 ui/desktop/src/platform/windows/bin/uv.exe
e21fbbaf67de8931eb0c8cfc8350e3a838537355 blob 48721448 ui/desktop/src/bin/goosed
c87175ea85f5b8c32b85245a6f6703d8cce489d3 blob 98101069 ui/desktop/src/bin/goose_ffi.dll

for source code tar ball

419M    documentation

Related: https://github.com/block/goose/issues/3914

[alexgleason]

This is crazy. I can't even clone goose anymore. Been stuck on 1% for like 5 minutes.

[jamadeo]

I looked into the main reasons why the goose repo is as large as it is. It might not be what you think!

The main offender is the rossjrw/pr-preview-action we have running to generate preview documentation websites for every PR that affects the docs. This is because it will do a full build of the documentation page and commit that to a new directory in the gh-pages branch. It does run some logic to "clean up" once the PR closes, but this clean up is simply a new commit to delete the files -- they're still in the git history.

Running git filter-repo --path pr-preview/ --invert-paths --refs origin/gh-pages shrinks .git from 2.2G -> 626M, and doesn't re-write any history outside of the gh-pages branch.

There are other large artifacts that bloat the repo of course, but this is an easy thing to fix!

I'm not sure what we'll want to do to fix it permanently though. The action doesn't seem to support deleting the previews from history.