Command 'vagrant up' fails with 'An error occurred in the underlying SSH library'

[ottok]

Some users encounter this bug with VirtualBox 6.1 on MacOS X:

An error occurred in the underlying SSH library that Vagrant uses. The error message is shown below. In many cases, errors from this library are caused by ssh-agent issues. Try disabling your SSH agent or removing some keys and try again.

If the problem persists, please report a bug to the net-ssh project.

timeout during server version negotiating

This seems to be some VirtualBox 6.1 compatibility issue.

The quick solution is to completely uninstall and remove VirtualBox 6.1 and install version 5.2 instead, which is known to be a good version and works well.

Related: #96

[JoosuaKoskinen]

I experienced the same issue on Windows while testing VirtualBox 6.1. I managed to fix it without installing an older version. I'll try to backtrace the steps to fixing it.

[lindqke]

Here's some more context from a Macbook Pro w/ Mac Os Catalina 10.15.4 where vagrant up fails like this.

$ vagrant --version
Vagrant 2.2.9
$ VBoxManage --version
6.1.8r137981
$ ruby -ropenssl -e 'puts OpenSSL::OPENSSL_VERSION'
LibreSSL 2.8.3
$ vagrant up --debug
...
DEBUG virtualbox_6_1:   - [1, "ssh", 2222, 22, "127.0.0.1"]
DEBUG ssh: Checking key permissions: /Users/redacted/.vagrant.d/boxes/seravo-VAGRANTSLASH-wordpress/20200130.0.0/virtualbox/vagrant_private_key
 INFO ssh: Attempting SSH connection...
 INFO ssh: Attempting to connect to SSH...
 INFO ssh:   - Host: 127.0.0.1
 INFO ssh:   - Port: 2222
 INFO ssh:   - Username: vagrant
 INFO ssh:   - Password? false
 INFO ssh:   - Key Path: ["/Users/redacted/.vagrant.d/boxes/seravo-VAGRANTSLASH-wordpress/20200130.0.0/virtualbox/vagrant_private_key"]
DEBUG ssh:   - connect_opts: {:auth_methods=>["none", "hostbased", "publickey"], :config=>false, :forward_agent=>true, :send_env=>false, :keys_only=>true, :verify_host_key=>:never, :password=>nil, :port=>2222, :timeout=>15, :user_known_hosts_file=>[], :verbose=>:debug, :logger=>#<Logger:0x00007fdddb09de98 @level=0, @progname=nil, @default_formatter=#<Logger::Formatter:0x00007fdddb09dda8 @datetime_format=nil>, @formatter=nil, @logdev=#<Logger::LogDevice:0x00007fdddb09dce0 @shift_period_suffix=nil, @shift_size=nil, @shift_age=nil, @filename=nil, @dev=#<StringIO:0x00007fdddb09df10>, @mon_mutex=#<Thread::Mutex:0x00007fdddb09dc18>, @mon_mutex_owner_object_id=70295419678320, @mon_owner=nil, @mon_count=0>>, :keys=>["/Users/redacted/.vagrant.d/boxes/seravo-VAGRANTSLASH-wordpress/20200130.0.0/virtualbox/vagrant_private_key"], :remote_user=>"vagrant"}
DEBUG ssh: == Net-SSH connection debug-level log START ==
DEBUG ssh: D, [2020-05-28T11:01:36.029234 #41462] DEBUG -- net.ssh.transport.session[3feef59a0be0]: establishing connection to 127.0.0.1:2222
D, [2020-05-28T11:01:36.029603 #41462] DEBUG -- net.ssh.transport.session[3feef59a0be0]: connection established
I, [2020-05-28T11:01:36.029699 #41462]  INFO -- net.ssh.transport.server_version[3feef59a9f88]: negotiating protocol version
D, [2020-05-28T11:01:36.029735 #41462] DEBUG -- net.ssh.transport.server_version[3feef59a9f88]: local is `SSH-2.0-Ruby/Net::SSH_5.2.0 x86_64-darwin19'

DEBUG ssh: == Net-SSH connection debug-level log END ==
 INFO interface: error: An error occurred in the underlying SSH library that Vagrant uses.
The error message is shown below. In many cases, errors from this
library are caused by ssh-agent issues. Try disabling your SSH
agent or removing some keys and try again.

If the problem persists, please report a bug to the net-ssh project.

timeout during server version negotiating
...

I redacted few thousand lines of debug output above. There's no other error messages than

timeout during server version negotiating

It's really hard to find the root cause for this. It might even be unrelated to net-ssh, which I guess is the "underlying SSH library that Vagrant uses".

I have tried with ssh-agent running, without it, with new clean keychain (where Mac stores passwords, keys, certs etc), removing and reinstalling VirtualBox and Vagrant with plenty of reboots in between and so on to no avail.

What I haven't tried is adding the Vagrant generated keys to ssh-agent. Nothing, except the error message, implies I should do that. ssh-add -l shows only my primary key (~/.ssh/id_rsa).

I've downgraded to VirtualBox 5.2 for now.

[spktklr]

I've had various SSH issues happen, and have witnessed others have them happen sporadically with Seravo's boxes over a period of time. Now experiencing it with a fresh 2020 MBP macOS 10.15.5. In comparison, https://github.com/roots/trellis ssh works on 6.x.

Circumventing with VB 5.2 for the time being.

[ottok]

On Linux Vagrant 2.2.x and VirtualBox 6.1.x it is confirmed to work OK on multiple computers and users. For Windows and Mac with VirtualBox 6.1.x the situation is not properly tested yet.

[JoosuaKoskinen]

I'm now experiencing this error with Vagrant 2.2.9, VirtualBox 6.1 and Ubuntu 20.04:

▶ vagrant up           
Bringing machine 'wordpress-box' up with 'virtualbox' provider...
==> wordpress-box: Checking if box 'seravo/wordpress-beta' version '20200606.0.0' is up to date...
==> wordpress-box: [vagrant-hostsupdater] Checking for host entries
==> wordpress-box: [vagrant-hostsupdater] Writing the following entries to (/etc/hosts)
==> wordpress-box: [vagrant-hostsupdater]   192.168.32.234  wordpress  # VAGRANT: 6a2281feaa9240133c3062746e5e9566 (wordpress-box) / 5b94e9c7-f17a-42df-ad72-61f1ab606e4c
==> wordpress-box: [vagrant-hostsupdater]   192.168.32.234  wordpress.local  # VAGRANT: 6a2281feaa9240133c3062746e5e9566 (wordpress-box) / 5b94e9c7-f17a-42df-ad72-61f1ab606e4c
==> wordpress-box: [vagrant-hostsupdater]   192.168.32.234  www.wordpress.local  # VAGRANT: 6a2281feaa9240133c3062746e5e9566 (wordpress-box) / 5b94e9c7-f17a-42df-ad72-61f1ab606e4c
==> wordpress-box: [vagrant-hostsupdater]   192.168.32.234  webgrind.wordpress.local  # VAGRANT: 6a2281feaa9240133c3062746e5e9566 (wordpress-box) / 5b94e9c7-f17a-42df-ad72-61f1ab606e4c
==> wordpress-box: [vagrant-hostsupdater]   192.168.32.234  adminer.wordpress.local  # VAGRANT: 6a2281feaa9240133c3062746e5e9566 (wordpress-box) / 5b94e9c7-f17a-42df-ad72-61f1ab606e4c
==> wordpress-box: [vagrant-hostsupdater]   192.168.32.234  mailcatcher.wordpress.local  # VAGRANT: 6a2281feaa9240133c3062746e5e9566 (wordpress-box) / 5b94e9c7-f17a-42df-ad72-61f1ab606e4c
==> wordpress-box: [vagrant-hostsupdater]   192.168.32.234  browsersync.wordpress.local  # VAGRANT: 6a2281feaa9240133c3062746e5e9566 (wordpress-box) / 5b94e9c7-f17a-42df-ad72-61f1ab606e4c
==> wordpress-box: [vagrant-hostsupdater]   192.168.32.234  info.wordpress.local  # VAGRANT: 6a2281feaa9240133c3062746e5e9566 (wordpress-box) / 5b94e9c7-f17a-42df-ad72-61f1ab606e4c
==> wordpress-box: [vagrant-hostsupdater] This operation requires administrative access. You may skip it by manually adding equivalent entries to the hosts file.
[sudo] password for joosua: 
==> wordpress-box: Running provisioner: shell...
An error occurred in the underlying SSH library that Vagrant uses.
The error message is shown below. In many cases, errors from this
library are caused by ssh-agent issues. Try disabling your SSH
agent or removing some keys and try again.

If the problem persists, please report a bug to the net-ssh project.

timeout during server version negotiating

[JoosuaKoskinen]

Works fine after running killall ssh-agent.

[niemenmaa]

Same problem on 5.6.15-1-MANJARO :

 vagrant --version
Vagrant 2.2.9

vboxmanage --version
6.1.8r137981

killall ssh-agent doesn't resolve the problem.

[ottok]

This should be fixed or at least mitigated in the latest seravo/wordpress-beta box version 20200610.0.0. Please test again.

[ottok]

Quick commands to test:

git clone https://github.com/Seravo/wordpress.git
cd wordpress
git checkout beta
vagrant box update
vagrant up

[nellileinonen]

Testing seravo/wordpress-beta box version 20200610.0.0.

I was able to to reproduce the issue with

5.6.15-1-MANJARO
vagrant --version
Vagrant 2.2.9
vboxmanage --version
6.1.8r137981

When downgrading the virtualbox 6.1.8. --> 6.1.6. everything worked ok. So this combination works

5.6.15-1-MANJARO
vagrant --version
Vagrant 2.2.9
vboxmanage --version
6.1.6r137129

[niemenmaa]

Can confirm. Issue persists with 6.1.8r137981vbox, but installing 6.1.6r137129 version fixes the issue on wordpress-beta box. SSH still hangs on vagrant up for sometime, but reconnects and finishes the booting process.

Thanks for the help @ottok and @nellileinonen !

[ottok]

Could you please re-test using the latest Vagrant box? ​ https://github.com/Seravo/wordpress/issues/96#issuecomment-740159046

​​A new Vagrant box version has been released: https://app.vagrantup.com/seravo/boxes/wordpress/versions/20201205.0.0

[elguitar]

Can anyone from @Seravo/dev-team reproduce this?

[JoosuaKoskinen]

I've been running VirtualBox 6.1.18 for a while now with Vagrant 2.2.14 and ssh-agent running with no issues.