Error“internal inconsistency” in downloading file from special sftp server whish is old cipher

[zjun9851]

• When testing sftp，appear this： "ssh: handshake failed: ssh: no common algorithm for client to server cipher; client offered: [[email protected] [email protected] aes128-ctr aes192-ctr aes256-ctr], server offered: [aes128-cbc 3des-cbc blowfish-cbc]"

• So, i add Ciphers to the clientConfig below, success to new sftp.Client ` auth := make([]ssh.AuthMethod, 0) auth = append(auth, ssh.Password(sftpOrFtpUrlInfo.Password)) var sshconfig ssh.Config sshconfig.SetDefaults() cipherOrder := sshconfig.Ciphers sshconfig.Ciphers = append(cipherOrder, "aes128-cbc", "3des-cbc", "blowfish-cbc") clientConfig := ssh.ClientConfig{ Config: sshconfig, User: sftpOrFtpUrlInfo.Username, Auth: auth, Timeout: 5 * time.Second, HostKeyCallback: ssh.InsecureIgnoreHostKey(), }

  addr := iptools.BuildAddr(sftpOrFtpUrlInfo.Ip, sftpOrFtpUrlInfo.Port) client, err := ssh.Dial("tcp", addr, &clientConfig)`

• but, Error “internal inconsistency” from WriteTo(dstFile) in downloading file from the sftp server.

• In fact， it is working from other sftp server.

Is there a way to solve this problem? Looking forward to your reply.

[eikenb]

In fact， it is working from other sftp server.

What is the difference between the server it is working on and the one that it isn't?

[zjun9851]

1. the special sftp server means whish is old cipher， is not working.
2. “other sftp server” means not appearing “ssh：handshake failed” without add ciphers， is working.

[zjun9851]

In fact， it is working from other sftp server.

What is the difference between the server it is working on and the one that it isn't? 1.the special sftp server means whish is old cipher， is not working. 2.“other sftp server” means not appearing “ssh：handshake failed” without add ciphers， is working.

[eikenb]

To rephrase to make sure I understand...

You have a newer SFTP server that supports the newer ciphers and an older SFTP server that doesn't. The one where you can use the new, default ciphers works but the old one where you must configure it to use different ciphers doesn't.

Does that sound right?

[zjun9851]

To rephrase to make sure I understand...

You have a newer SFTP server that supports the newer ciphers and an older SFTP server that doesn't. The one where you can use the new, default ciphers works but the old one where you must configure it to use different ciphers doesn't.

Does that sound right?

Yes. the old one can connect and upload file, but Error “internal inconsistency” from WriteTo(dstFile) in downloading file

[eikenb]

I'm going to try to reproduce this by modifying the config for an openssh server to not support any of the default supported ciphers. I'm going to focus on getting the ssh connection working first, then see about getting sftp to work over that.

[zjun9851]

tks. Looking forward to your reply.

[eikenb]

I was looking into which ciphers to use and saw this...

https://github.com/golang/crypto/blob/22d7a77e9e5f409e934ed268692e56707cd169e5/ssh/cipher.go#L97-L99

[edit] thought that was supposed to display those lines... anyway, here is the quote that I was referencing above..

// cipherModes documents properties of supported ciphers. Ciphers not included
// are not supported and will not be negotiated, even if explicitly requested in
// ClientConfig.Crypto.Ciphers.

[eikenb]

Regarding the comment in ciphers.go mentioned above... the list of valid ciphers does not include any of the ones you added. Can you retest with any of the ciphers listed in that file?

[zjun9851]

Regarding the comment in ciphers.go mentioned above... the list of valid ciphers does not include any of the ones you added. Can you retest with any of the ciphers listed in that file?

as you said, i retest with any of the ciphers "aes128-cbc" or "3des-cbc" which listed in that file, but the error remains。

my code append "aes128-cbc" or "3des-cbc" or "aes128-cbc" , "3des-cbc": var sshconfig ssh.Config sshconfig.SetDefaults() cipherOrder := sshconfig.Ciphers sshconfig.Ciphers = append(cipherOrder, "aes128-cbc","3des-cbc") clientConfig := ssh.ClientConfig{ Config: sshconfig, User: sftpOrFtpUrlInfo.Username, Auth: auth, Timeout: 5 * time.Second, HostKeyCallback: ssh.InsecureIgnoreHostKey(), }

[eikenb]

@zjun9851 thanks for testing that. I'll let you know if I find anything in my testing. Sorry things are taking some time, I just started a new job and haven't had a lot of time to look into this.

[eikenb]

Just realized they kind of obfuscate the names of a few of the ciphers behind variable names that don't match up. Not sure why, but that was why I had questioned your cipher choice. I see now what was going on. Sorry about that.

I tested changing my openssh server to use Ciphers 3des-cbc,aes128-cbc and configured a simple crypto/ssh test program to use those same ciphers and got the error...

ssh: handshake failed: EOF

I also tried connecting with the openssh client using ssh -c 3des-cbc localhost and it connected fine. So there is definitely something up with the ssh library's handshake negotiation.

[eikenb]

This crypto/ssh issue seems like it might be related, but they pretty much say it should just work.

https://github.com/golang/go/issues/20201

[eikenb]

I just filed a ticket against x/crypto/ssh as I'm not sure what's wrong. We'll see what comes of it...

https://github.com/golang/go/issues/32075

[zjun9851]

This crypto/ssh issue seems like it might be related, but they pretty much say it should just work.

golang/go#20201

From my test, it just work in connecting and uploading file to the serverce, but Error “internal inconsistency” in downloading file from client.WriteTo(dstFile).

[eikenb]

@zjun9851 Would it be possible to post a code snippet showing how you connect in the case where it works? Maybe using the snippet I posted to https://github.com/golang/go/issues/32075 as a starting point. I just want to see the ssh connection part, because in my tests I wasn't able to make the ssh connection at all with the older ciphers.

Thanks.

[zjun9851]

@eikenb like this, `func OpenSftpClient(sftpOrFtpUrlInfo SftpOrFtpUrlInfo) (*sftp.Client, error) { auth := make([]ssh.AuthMethod, 0) auth = append(auth, ssh.Password(sftpOrFtpUrlInfo.Password)) var sshconfig ssh.Config sshconfig.SetDefaults() cipherOrder := sshconfig.Ciphers sshconfig.Ciphers = append(cipherOrder, "aes128-cbc", "3des-cbc") clientConfig := ssh.ClientConfig{ Config: sshconfig, User: sftpOrFtpUrlInfo.Username, Auth: auth, Timeout: 5 * time.Second, HostKeyCallback: ssh.InsecureIgnoreHostKey(), }

addr := iptools.BuildAddr(sftpOrFtpUrlInfo.Ip, sftpOrFtpUrlInfo.Port)
client, err := ssh.Dial("tcp", addr, &clientConfig)
if err != nil {
	return nil, err
}
sftpClient, err := sftp.NewClient(client)
if err != nil {
	client.Close()
	return nil, err
}
return sftpClient, nil

}`

[zjun9851]

@eikenb And the download file code like this: `func DownloadAFileFromSftp(localFile string, sftpOrFtpUrlInfo SftpOrFtpUrlInfo) (bool, error) { var ( err error sftpClient *sftp.Client ) sftpClient, err = OpenSftpClient(sftpOrFtpUrlInfo) if err != nil { return false, err } defer sftpClient.Close() srcFile, err := sftpClient.Open(sftpOrFtpUrlInfo.RemoteFilePath) if err != nil { return false, err } defer srcFile.Close() dstFile, err := os.Create(localFile) if err != nil { return false, err } defer dstFile.Close()

if _, err = srcFile.WriteTo(dstFile); err != nil {
	return false, err
}
return true, nil

}`

[zjun9851]

@eikenb and my testing sftp servers 's OS info is below， [root@xxx Desktop]# cat /proc/version Linux version 2.6.32-504.16.2.el6.x86_64 ([email protected]) (gcc version 4.4.7 20120313 (Red Hat 4.4.7-9) (GCC) ) #1 SMP Tue Mar 10 17:01:00 EDT 2015 [root@xxx Desktop]# ssh -V OpenSSH_5.3p1, OpenSSL 1.0.1e-fips 11 Feb 2013 [root@xxx Desktop]#

[eikenb]

Hey @zjun9851, sorry for the delay in responding. I started a new job a little while back and have been super busy getting up to speed.

It looks to me like you are just connecting to the server using one of the modern ciphers. Downloading OpenSSH 5.3, it supports "aes128-ctr,aes192-ctr,aes256-ctr" out of the box and your client connection is probably using one of those. In your example, you include all the default ciphers...

var sshconfig ssh.Config
sshconfig.SetDefaults()
cipherOrder := sshconfig.Ciphers
sshconfig.Ciphers = append(cipherOrder, "aes128-cbc", "3des-cbc")

SetDefaults() sets the config.Ciphers to preferredCiphers which is defined as...

var preferredCiphers = []string{
    "[email protected]",
     chacha20Poly1305ID,
    "aes128-ctr", "aes192-ctr", "aes256-ctr",
}

So you are including the modern ciphers. You also don't mention that you are restricting the ciphers on the server side to only old ciphers. So I think in your example you are connecting via one of the aes*-ctr ciphers.

[adamdecaf]

I ran into this when using https://github.com/atmoz/sftp as the SFTP server, but I couldn't find a cipher (or key exchange) combo that fixed the intermediate issues. Instead my service retries the file download a couple times, which works a lot better. I can help test some fixes if anyone has ideas.

[eikenb]

Hey @adamdecaf,

Thanks for the offer, but as far as I can tell this looks like an upstream issue with the ssh library. I've filed an issue about it. https://github.com/golang/go/issues/32075