prose icon indicating copy to clipboard operation
prose copied to clipboard

Very large git size

Open lgrcia opened this issue 2 years ago • 5 comments

When cloning prose, the repository weights 83Mb, but the actual code base is only ~10Mb (including notebooks stored in docs). I know this is coming from old voluminous files versioned that I would be happy to remove from git history (we are talking about versions < 1.0).

lgrcia avatar Jul 18 '23 09:07 lgrcia

You could use the command git gc --aggressive --prune=now (see doc). But 83MB is not an especially big repo, IMHO.

onekiloparsec avatar Mar 17 '25 14:03 onekiloparsec

Thanks for the idea @onekiloparsec! I tried with many other things and I could not really get rid of the large files. I'd need to give it another try! Thanks again

lgrcia avatar Mar 17 '25 15:03 lgrcia

One of the strength of git is ensure data consistency throughout history. Logically, it is very hard to "remove" anything from its data. Again, I'd be happy to help further if you explain a bit more why you think this repo is big. :-) (Not necessarily a reference, but I've worked with 10GB+ repos. It's a pain with the first clone, but after that, it's fine).

onekiloparsec avatar Mar 17 '25 15:03 onekiloparsec

Sure! Sometimes prose has to be cloned and tested in very remote places (like Antarctica) with very limited bandwidth. Also the big files were mostly part of erroneous commits, mostly mistakes.

lgrcia avatar Mar 17 '25 20:03 lgrcia

Wow, interesting. Indeed, Antarctica isn't probably high-speed connexion.

I just stumbled upon this, highly starred: https://github.com/rtyley/bfg-repo-cleaner

Looks quite easy to use.

onekiloparsec avatar Mar 17 '25 20:03 onekiloparsec