waveterm icon indicating copy to clipboard operation
waveterm copied to clipboard

Update to 0.7.x - Unable to SSH to Machines

Open oliv10 opened this issue 11 months ago • 20 comments

Describe the bug Unable to ssh into any machine. Reverting to an older version allows the application to SSH into other machines without issue.

To Reproduce Steps to reproduce the behavior: Update to version 0.7.x from 0.6.x

Expected behavior ssh to work correctly.

Screenshots If applicable, add screenshots to help explain your problem.

Screenshot 2024-03-19 at 10 12 37 AM Screenshot 2024-03-19 at 10 18 20 AM

Desktop (please complete the following information):

  • OS: MacOS M1 Max
  • Version 0.7.1

Additional context Valid ssh sessions with no changes to my known_hosts file.

Clearing the known_hosts file did not make any changes to this issue.

Screenshot 2024-03-19 at 10 20 00 AM
Screenshot 2024-03-19 at 10 20 16 AM

oliv10 avatar Mar 19 '24 14:03 oliv10

Interesting--I would have expected clearing the known_hosts to at least temporarily make it go away. I'm really hoping it isn't an underlying bug in the library we're using. In the meantime, there are a few other things we could look at.

Which keywords are you using for this connection in your ssh_config file? You don't need to tell me anything like user and/or hostname, but it's possible there are more obscure ones that we don't support yet.

If you take a look at the known_hosts file, could you check what type of key is causing the error? (It should say before the key itself)

oneirocosm avatar Mar 20 '24 02:03 oneirocosm

Same here after an update but different error. Ubuntu 22.04

prompt> error, invalid packet received from waveshell client: raw[which: no mshell-v0.5 in (/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/home/stheil/.mshell)]

stheil15 avatar Mar 20 '24 08:03 stheil15

@oneirocosm

Interesting--I would have expected clearing the known_hosts to at least temporarily make it go away. I'm really hoping it isn't an underlying bug in the library we're using. In the meantime, there are a few other things we could look at.

Clearning the known hosts does make it go away, I typed that sentence then testing it again for sanity's sake (forgot to update the sentence), and it did solve the issue. But it was kinda hit or miss with if it worked or not because the first time it did not fix it, which is very strange to me.

Which keywords are you using for this connection in your ssh_config file? You don't need to tell me anything like user and/or hostname, but it's possible there are more obscure ones that we don't support yet.

Nothing that the 0.6.x version didn't support. I am truly unsure what caused the issue to arise all of the sudden.

oliv10 avatar Mar 20 '24 15:03 oliv10

Screenshot 2024-03-20 at 12 03 18 PM

After removing the offending lines (one of which was the machine I am attempting to connect to) from the known hosts file it then throws this error. Below are the types of keys that it was angry at.

Screenshot 2024-03-20 at 12 03 26 PM

oliv10 avatar Mar 20 '24 16:03 oliv10

@stheil15

Same here after an update but different error. Ubuntu 22.04

prompt> error, invalid packet received from waveshell client: raw[which: no mshell-v0.5 in > (/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/home/stheil/.mshell)]

This looks like a separate issue that I've seen in development builds but not in production. You may be able to get around it by explicitly clicking the reinstall button and waiting a few minutes. (We unfortunately don't have a good install indicator when this happens).

oneirocosm avatar Mar 20 '24 21:03 oneirocosm

@oliv10

Nothing that the 0.6.x version didn't support. I am truly unsure what caused the issue to arise all of the sudden.

We made a bunch of ssh changes in this update. Rather than using a text-based system with the regular ssh command, we're now using golang's x/crypto/ssh library. So there's a chance that these updates broke something that worked in the previous version. I imagine things like HashKnownHosts and FingerprintHash could cause some problems until more support is added.

oneirocosm avatar Mar 20 '24 21:03 oneirocosm

Granted, none of that directly fixes your issue. Did you by chance get a popup with the remote host identification has changed error? Typically that would mean, that waveterm is finding a different entry in ~/.ssh/known_hosts, ~/.ssh/known_hosts2, /etc/ssh/known_hosts, or /etc/ssh/known_hosts2 for that connection, but the credentials from that file don't match what is being received. But I would expect that you'd also see this when connecting with a regular ssh command as well.

oneirocosm avatar Mar 20 '24 21:03 oneirocosm

I am getting the remote host identification has changed error. However, when I ssh into the machine using my standard terminal it works just fine without issue. No other known_hosts files exist on my system either.

oliv10 avatar Mar 20 '24 23:03 oliv10

Would you happen to have two entries for the same remote in your known_hosts file but with different keys?

oneirocosm avatar Mar 21 '24 00:03 oneirocosm

If so, it could be related to this bug https://github.com/golang/go/issues/36126

oneirocosm avatar Mar 21 '24 00:03 oneirocosm

Would you happen to have two entries for the same remote in your known_hosts file but with different keys?

Not after removing the three keys it was mad about.

After wiping the known_hosts in its entirety and ssh into it using my normal terminal, it creates three known hosts for it using three different key types. So, I don't think that should be an issue anyway.

oliv10 avatar Mar 21 '24 00:03 oliv10

If it is creating the three known hosts entries, it likely does come from the bug I linked above. It seems like the library we're using can't handle being provided multiple key types. There is a PR to fix it, but it looks like it hasn't been touched in a while. Worst case scenario, we might be able to fork the library and implement the fix ourselves—at least until it gets merged into the original library.

oneirocosm avatar Mar 21 '24 01:03 oneirocosm

Well, its not. Cause with the known_hosts file having just those three keys that it generated from ssh into it using the standard terminal. Switching back to Wave with just those created worked perfectly fine.

oliv10 avatar Mar 21 '24 01:03 oliv10

Screenshot 2024-03-20 at 9 47 45 PM

known_hosts file

Screenshot 2024-03-20 at 9 48 40 PM

Connection alive and happy, sans a few unrelated errors.

oliv10 avatar Mar 21 '24 01:03 oliv10

Screenshot 2024-03-20 at 9 49 54 PM

Executes commands as it would through any other terminal

oliv10 avatar Mar 21 '24 01:03 oliv10

Oh good—I'm glad you have a workaround! Still, that means that we aren't writing to known_hosts properly. If you noticed any glaring differences (that don't require you to post your keys) between the known_hosts we produced and the known_hosts ssh produced, that would be helpful!

oneirocosm avatar Mar 21 '24 19:03 oneirocosm

Oh good—I'm glad you have a workaround! Still, that means that we aren't writing to known_hosts properly. If you noticed any glaring differences (that don't require you to post your keys) between the known_hosts we produced and the known_hosts ssh produced, that would be helpful!

With the known_hosts empty and trying to connect using wave it wouldn't prompt me to create the new keys at all. So I would say that's probably the first thing to take a look at.

oliv10 avatar Mar 22 '24 00:03 oliv10

I have the same problems. Ubuntu machines. It seams like there isn't any new line after the key is added to known_hosts. So the first one is ok. When the second is added it just appends to the first one. Manually editing the row and press enter to force a newline makes everything correct again.

psgandalf avatar Apr 18 '24 20:04 psgandalf

I see where the issue is now. And it shouldn't be too hard to fix, so it will be in our next release.

oneirocosm avatar Apr 18 '24 21:04 oneirocosm

a linux build confirmes that the fix is the solution to the problem. Thx for the quick fix.

psgandalf avatar Apr 20 '24 07:04 psgandalf

While this fixed part of this issue, it seems the initial issue still exists as referenced here https://github.com/golang/go/issues/36126

Since that one is more active, I will close this one and continue to work on a fix there.

oneirocosm avatar Jul 05 '24 21:07 oneirocosm