Cloning big repos fails

[jachro]

It looks there are at least two repositories in Renkulab that triples-generator cannot clone: https://renkulab.io/gitlab/luis.salamanca/democrasci_preprocwp1 and https://renkulab.io/gitlab/david.sollander/windtunnelmotus (in fact, I cannot clone them on my laptop either). The error for both projects is slightly different but I think, the reason might be the same.

$ git clone https://renkulab.io/gitlab/luis.salamanca/democrasci_preprocwp1.git
Cloning into 'democrasci_preprocwp1'...
remote: Enumerating objects: 15148, done.
error: RPC failed; curl 92 HTTP/2 stream 0 was not closed cleanly: INTERNAL_ERROR (err 2)
error: 22 bytes of body are still expected
fetch-pack: unexpected disconnect while reading sideband packet
fatal: early EOF
fatal: fetch-pack: invalid index-pack output

$ git clone https://renkulab.io/gitlab/david.sollander/windtunnelmotus.git
Cloning into 'windtunnelmotus'...
remote: Enumerating objects: 137, done.
remote: Counting objects: 100% (137/137), done.
error: RPC failed; curl 16 Error in the HTTP2 framing layer
error: 5 bytes of body are still expected
fetch-pack: unexpected disconnect while reading sideband packet
fatal: early EOF
fatal: fetch-pack: invalid index-pack output

I found this stackoverflow thread listing these 4 potential reasons:

1. Slow Internet. Switching to LAN cable with stable network connection helps in many cases. Avoid doing any parallel network intensive task while you are fetching.
2. Small TCP/IP connection time out on Server side from where you are trying to fetch. Not much you can do about. All you can do is request your System Admin or CI/CD Team responsible to increaseTCP/IP Timeout and wait.
3. Heavy Load on Server. Due to heavy server load during work hour downloading a large file can fail constantly.Leave your machine after starting download for night.
4. Small HTTPS Buffer on Client machine. Increasing buffer size for post and request might help but not guaranteed.

git config --global http.postBuffer 524288000
git config --global http.maxRequestBuffer 524288000
git config --global core.compression 0

I've tried option 4 already but it didn't get me anywhere.

[olevski]

Probably related to this: https://docs.gitlab.com/ee/topics/git/troubleshooting_git.html#git-clone-over-http-fails-with-transfer-closed-with-outstanding-read-data-remaining-error

[olevski]

In addition to increasing the http.postBuffer setting, it is also recommended that nginx['client_max_body_size'] is increased as well. See: https://stackoverflow.com/questions/6842687/the-remote-end-hung-up-unexpectedly-while-git-cloning