concurrent calls to `git` cause issues with encrypted SSH keys

[Thermi]

Hello,

glide install does not work for me, because the Git repositories it needs to pull from are accessible over SSH. I am authenticating against it with my SSH key. The key's private key is encrypted. The concurrent calls that glide install makes, cause issues with the passphrase dialogue on the shell. Only the password for the first password request can be entered. Further attempts just echo the entered on the shell. Hitting enter just causes each error message to appear. Please fix it. Decrypting my private key is not an option.

glide install
[WARN] The name listed in the config file (github.com/<private repo>) does not match the current location (.)
[INFO] Downloading dependencies. Please wait...
[INFO] Fetching updates for github.com/dgrijalva/jwt-go.
[INFO] Fetching updates for github.com/elazarl/goproxy.
[INFO] Fetching updates for github.com/go-kit/kit.
[INFO] Fetching updates for github.com/kr/logfmt.
[INFO] Fetching updates for github.com/go-stack/stack.
[INFO] Fetching updates for github.com/<private repo>
[INFO] Fetching updates for gopkg.in/mgo.v2.
[INFO] Fetching updates for github.com/<private repo>
[INFO] Fetching updates for github.com/go-logfmt/logfmt.
Enter passphrase for key '/home/thermi/.ssh/id_rsa_git': Enter passphrase for key '/home/thermi/.ssh/id_rsa_git': Enter passphrase for key '/home/thermi/.ssh/id_rsa_git': Enter passphrase fo
r key '/home/thermi/.ssh/id_rsa_git': Enter passphrase for key '/home/thermi/.ssh/id_rsa_git': Enter passphrase for key '/home/thermi/.ssh/id_rsa_git': Enter passphrase for key '/home/thermi
/.ssh/id_rsa_git': Enter passphrase for key '/home/thermi/.ssh/id_rsa_git': 


[WARN] Unable to checkout github.com/elazarl/goproxy
[ERROR] Update failed for github.com/elazarl/goproxy: Klone nach '<private path>/vendor/github.com/elazarl/goproxy' ...
Permission denied (publickey).
fatal: Could not read from remote repository.

Please make sure you have the correct access rights
and the repository exists.
: exit status 128


[WARN] Unable to checkout github.com/<private repo>
[ERROR] Update failed for github.com/<private repo>: Klone nach '<private path>/vendor/<private repo>' ...
Permission denied (publickey).
fatal: Could not read from remote repository.

Please make sure you have the correct access rights
and the repository exists.
: exit status 128


[WARN] Unable to checkout github.com/go-stack/stack
[ERROR] Update failed for github.com/go-stack/stack: Klone nach '<private path>/vendor/github.com/go-stack/stack' ...
Permission denied (publickey).
fatal: Could not read from remote repository.

Please make sure you have the correct access rights
and the repository exists.
: exit status 128


[WARN] Unable to checkout github.com/go-kit/kit
[ERROR] Update failed for github.com/go-kit/kit: Klone nach '<private path>/vendor/github.com/go-kit/kit' ...
Permission denied (publickey).
fatal: Could not read from remote repository.

Please make sure you have the correct access rights
and the repository exists.
: exit status 128

[WARN] Unable to checkout github.com/<private path>
[ERROR] Update failed for github.com/<private path>: Klone nach '<private path>/vendor/github.com/<private repo>' ...
Permission denied (publickey).
fatal: Could not read from remote repository.

Please make sure you have the correct access rights
and the repository exists.
: exit status 128


[WARN] Unable to checkout github.com/dgrijalva/jwt-go
[ERROR] Update failed for github.com/dgrijalva/jwt-go: Klone nach '<private path>/vendor/github.com/dgrijalva/jwt-go' ...
Permission denied (publickey).
fatal: Could not read from remote repository.

Please make sure you have the correct access rights
and the repository exists.
: exit status 128


[WARN] Unable to checkout github.com/go-logfmt/logfmt
[ERROR] Update failed for github.com/go-logfmt/logfmt: Klone nach '<private path>/vendor/github.com/go-logfmt/logfmt' ...
Permission denied (publickey).
fatal: Could not read from remote repository.

Please make sure you have the correct access rights
and the repository exists.
: exit status 128
[INFO] Downloading dependencies. Please wait...
[INFO] Setting references.
[ERROR] Failed to set version on github.com/dgrijalva/jwt-go to 40bd0f3b4891a9d7f121bfb7b8e8b0525625e262: open /<private path>: no such file or directory
[ERROR] Failed to set version on github.com/elazarl/goproxy to 970f4ed8995ab98f808e4abf06f52660aeaec7a9: open <private path>: no such file or directory
[ERROR] Failed to set version on github.com/go-kit/kit to bf1de9351592667fd8739200fa355b6a5c53adbf: open <private path>: no su
ch file or directory
[ERROR] Failed to set version on github.com/go-logfmt/logfmt to a0ff333161fe5c2daed0ba52d1792bd3d2531b94: open<private path>: no such file or directory
[ERROR] Failed to set version on github.com/go-stack/stack to ac4c579c437a23e1a15814c867ccb82122fb383b: open <private path>: no such file or directory
[ERROR] Failed to set version on github.com/<private repo> to 44f2828e41e3c8fff13882990585b9705c5e944f: open<private path>: no 
such file or directory
[ERROR] Failed to set version on github.com/<private repo> to 328fbec767d031d163970074bdd5529c4252e966: open <private path>: no such file or directory
[INFO] Setting version for github.com/kr/logfmt to b84e30acd515aadc4b783ad4ff83aff3299bdfe0.

[INFO] Setting version for gopkg.in/mgo.v2 to b6e2fa371e64216a45e61072a96d4e3859f169da.
An Error has occurred

[mattfarina]

I have a feeling this is due to concurrently calling Git.

[sdboyer]

@mattfarina seems like we should be able to add a flag for --serial-fetch or something, no?

I'll make a note of this in vsolver - I haven't implemented parallel fetch there yet, anyway, so the timing of this report is good.

[nubunto]

Also, not sure if this is related:

Glide version (glide --version): glide version 0.10.2

When I run glide install, the following happens:

[INFO] Lock file (glide.lock) does not exist. Performing update.
[INFO] Downloading dependencies. Please wait...
[INFO] Fetching updates for github.com/julienschmidt/httprouter.
[INFO] Fetching updates for github.com/pkg/errors.
[INFO] Fetching updates for github.com/thermeon/go-errors.
[INFO] Fetching updates for github.com/thermeon/go-logs.
Username for 'https://github.com': Username for 'https://github.com': nubunto
Password for 'https://[email protected]': Password for 'https://**MY PASSWORD GOES HERE AFTER I TYPE IT**@github.com': 
[WARN] Unable to checkout github.com/thermeon/go-logs
[ERROR] Update failed for github.com/thermeon/go-logs: Cloning into '/home/bruno/go-workspace/src/github.com/thermeon/forwarding-service/vendor/github.com/thermeon/go-logs'...
remote: Invalid username or password.
fatal: Authentication failed for 'https://github.com/thermeon/go-logs/'
: exit status 128


[WARN] Unable to checkout github.com/thermeon/go-errors
[ERROR] Update failed for github.com/thermeon/go-errors: Cloning into '/home/bruno/go-workspace/src/github.com/thermeon/forwarding-service/vendor/github.com/thermeon/go-errors'...
remote: Invalid username or password.
fatal: Authentication failed for 'https://github.com/thermeon/go-errors/'
: exit status 128
[ERROR] Failed to do initial checkout of config: Cloning into '/home/bruno/go-workspace/src/github.com/thermeon/forwarding-service/vendor/github.com/thermeon/go-logs'...
remote: Invalid username or password.
fatal: Authentication failed for 'https://github.com/thermeon/go-logs/'
: exit status 128
Cloning into '/home/bruno/go-workspace/src/github.com/thermeon/forwarding-service/vendor/github.com/thermeon/go-errors'...
remote: Invalid username or password.
fatal: Authentication failed for 'https://github.com/thermeon/go-errors/'
: exit status 128

[nubunto]

~~Some of those are private repositories, FWIW~~ All of the failing packages are private repositories.

[ironcladlou]

As a workaround, can you add the offending key(s) to your SSH agent?

[hasufell]

As a workaround, can you add the offending key(s) to your SSH agent?

That's quite a use case assumption. SSH agent is a security problem and not everyone wants to use it.

[mattfarina]

Does dep handle this well?

[sdboyer]

we don't right now, no. we do have a logical choke point we could exploit to enforce it relatively easily, but there hasn't been a direct complaint about it yet over there, so I've deferred it in favor of other things.

[mattfarina]

@sdboyer have you thought about what a good solution would be?

[sdboyer]

The only one i've been able to come up with doesn't necessarily qualify as "good" - probably more like "passable" 😄

We use the aforementioned choke point to throttle down all network activity to the point where all network calls are serialized. An environment variable (preferred) or flag (i'd rather not increase the surface area of the interface for this) then determines whether the throttling behavior is active.

[luigi-riefolo]

any update?