atlantis
atlantis copied to clipboard
runatlantis
Implement shallow repo cloning in merge checkout strategy
When the repository is quite large clones are becoming slower. Previously Atlantis fetched full repository in --checkout-strategy=merge mode. However, that's not required, we only should have enough commits in our clone to perform a merge between base and feature branches, not full history.
This implements rough guessing: fetch 50 (user-configurable) commits from each of base and feature branches, and if their merge base is not in these commits, fetch full repo history. This is not ideal solution and could be improved (e.g. increase the number of fetched commits in several attempts before full fetch), but I think it should be good enough in most cases.
I'm not sure about the flag name and enabling this behaviour by default. The flag could be renamed to something like --checkout-merge-depth, if that would be preferable. Also, I could change the default value to zero and treat it as "always fetch full repo", to comply with previous behaviour.
Huh, come to think of it, implementing cache for git repo would probably be a better solution. Something among these lines:
- Add flag --enable-repo-caching
- If flag enabled, clone bare repo to some workdir (e.g. data_dir/repos/owner/repo_name/repo) on first usage
- Clone to PR workdir from this bare repo
- On each subsequential clone fetch all remote branches in bare repo and then clone/fetch to workspaces
Current PR would still probably be useful in cases when there are tons of files in repo and even cloning from dir to dir takes a long time, but that's more rare scenario.
Could we reopen this? I think this has a lot of value for repos with many commits
go for it.
On Tue, Nov 22, 2022, 9:18 p.m. nitrocode @.***> wrote:
Could we reopen this? I think this has a lot of value for repos with many commits
— Reply to this email directly, view it on GitHub https://github.com/runatlantis/atlantis/pull/2387#issuecomment-1324565868, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAQ3ERBDMVEC7IYJB6LXR3DWJWSL5ANCNFSM53PH5H4Q . You are receiving this because you modified the open/close state.Message ID: @.***>
@jamengual i cannot open this for some reason. The branch still exists. The button is unclickable.
Ah it's because the master branch was renamed to main. I recreated master from main and now I can reopen this PR.
@jilyaluk please resolve conflicts when time permits. We appreciate your patience