gh-clone-org icon indicating copy to clipboard operation
gh-clone-org copied to clipboard

Published 20 hours ago • verified publisher icon matt-bartel

Make script faster with shallow cloning and parallel

Open rmartine-ias opened this issue 8 months ago • 0 comments

This script is quite slow for my organization, which has ~2k repositories.

This PR has two changes that speed it up significantly:

  1. Add an option to shallow-clone repositories. Most of the time, I want to clone the org so I can learn things about the current state, and I don't need the whole history. This significantly decreases time to clone, and disk space used. I don't know if this is the most common use case, so I made it opt-in. Happy to change this to being the default if you want.

  2. Use GNU parallel if it's available. I haven't had issues with -j 10 on my machine (I have a similar script that uses this), but I'm happy to change it to something configurable if you want. This gives around a 10x speedup.

I have various other potential improvements in mind, that I use in my own script, but I wanted to get your feedback on these first. Thank you for maintaining this script!

rmartine-ias avatar May 31 '24 20:05 rmartine-ias