FluentFTP
FluentFTP copied to clipboard
robinrodricks
TLS session resumption missing / Authentication failed because the remote party has closed the transport stream / Received an unexpected EOF or 0 bytes from the transport stream
FTP OS: Windows
FTP Server: FileZilla Server
Computer OS: Windows
I'm using FluentFTP in my application. (Great library by the way.) My application is multi-threaded, with each thread having its own instance of FtpClient but with the same ftp credentials, all connecting to the same FTP server just doing different file-copy or directory listing tasks. All works fine with FTP. I've recently done some work on extending this to work with FTPS. Again, works very nicely using a single thread, but as soon as I start using 2 threads (so 2 simultaneous connections) I'm getting an IOException:
Authentication failed because the remote party has closed the transport stream.
On the FTP server (FileZilla Server 0.9.60) I'm seeing this:
(036840)22/06/2018 10:55:19 - brian (10.0.0.26)> 450 TLS session of data connection has not resumed or the session does not match the control connection
I think (and this is pushing the bounds of my knowledge) that the message on the server indicates that the client does not support TLS session resumption. I gather that TLS session resumption acts to prevent a hacker from connecting to the data port before the client does - if the TLS session of the data connection matches the session of the control connection then both the client and server know that the data connection is genuine, whereas a mismatch indicates a potential attack.
I know that I can turn off TLS session resumption in the FTP server - but I don't want to as it is there to help make the server more secure. Falling back to a single connection is a viable workaround, but since it is faster to use multiple connections it would be nice to find a better way forward.
So: is there a setting that I am missing in FluentFTP to overcome this error?
Thank you.
Logs :
[fluentftp-log.txt](https://github.com/robinrodricks/FluentFTP/files/2127366/fluentftp-log.txt)
[ftp-server-log.txt](https://github.com/robinrodricks/FluentFTP/files/2127362/ftp-server-log.txt)
Nice description, but no, never heard of this. Seems like the server is kicking you because you have multiple connections from the same source IP. Maybe try setting a source IP limit (client.ActivePorts) and see if that helps.
I've found the relevant setting on the server - Under "FTP over TLS settings" there is the option:
[x] Require TLS session resumption on data connection when using PROT P.
That setting is checked by default. Unchecking it and all is fine, I can have as many ftps connections as I like. Checking it and it falls over again straight away with the same error. If I use the FileZilla client it seems happy with multiple connections regardless of the setting. So I'm assuming that FluentFTP doesn't support session resumption. I can live with that. Multiple ftps connections was always a nice to have.
Hopefully this information will be useful if anyone else encounters the same issue.
BTW, when connection I'm using client.DataConnectionType = FtpDataConnectionType.AutoPassive; so I'm not sure if client.ActivePorts is applicable?
Can confirm that this is the issue on Win 10 with latest filezila server 0.9.60.2
Connecting with
`using (var ftpClient = new FtpClient("127.0.0.1", 21, "user", "pass"))
{
ftpClient.EncryptionMode = FtpEncryptionMode.Explicit;
ftpClient.ValidateAnyCertificate = true; // TODO: test purporses
ftpClient.SslProtocols = SslProtocols.Tls12;
await ftpClient.ConnectAsync();
var directoryListing = await ftpClient.GetListingAsync();
Assert.Single(directoryListing.Select(dl => dl.Name));
}`
fails with
System.IO.IOException : Received an unexpected EOF or 0 bytes from the transport stream.
Stack Trace:
at System.Threading.Tasks.Task.Wait(Int32 millisecondsTimeout, CancellationToken cancellationToken)
at System.Threading.Tasks.Task.Wait()
at FluentFTP.FtpSocketStream.ActivateEncryption(String targethost, X509CertificateCollection clientCerts, SslProtocols sslProtocols)
at FluentFTP.FtpClient.OpenPassiveDataStream(FtpDataConnectionType type, String command, Int64 restart)
at FluentFTP.FtpClient.OpenDataStream(String command, Int64 restart)
in case Require TLS session resumption on data connection when using PROT P. is checked
@robinrodricks I don't think it's a server issue, it's something (TLS session resumption) that FluentFTP doesn't implement.
Seems like the server is kicking you because you have multiple connections from the same source IP.
This isn't what is happening. Even with a single connection, I get the same problem. The server log clearly indicates what's wrong:
450 TLS session of data connection has not resumed or the session does not match the control connection
Ideally, FluentFTP should correctly implement TLS session resumption. At the very least, it should consistently throw an exception, which is currently not the case. When I try to list a directory, I usually get an empty listing and no error, but every now and then, it throws an exception with the message above.
In any case, this issue should be reopened.
This bug is due to Fluent FTP is not support "Require TSL session resumption on data connection when using FTP over TLS"
Uncheck this item under FTP over TLS Settings on FileZilla Server. Have a try!
Is there any update on this? Later versions of FileZilla have removed the ability to not require TLS resumption and insist that it is absolutely necessary to create a secure connection. Note that other FTP clients (e.g. WinSCP) can connect to this same FTP server and seem to handle TLS resumption from the same computer to the same server.
Later versions of FileZilla have removed the ability to not require TLS resumption and insist that it is absolutely necessary to create a secure connection. Note that other FTP clients (e.g. WinSCP) can connect to this same FTP server and seem to handle TLS resumption from the same computer to the same server.
Agreed and taken note of.
Work in progress, sorry no news yet
@MarkDynna I have set up a Filezilla server here (FileZilla Server 1.5.1). I am using the current FluentFTP V40.0.0 (master) and am connecting to this server via TLS 1.2:
<Date> Info [Type] Message
<01.10.2022 20:22:36> FTP Session 8 127.0.0.1 [Response] 220-FileZilla Server 1.5.1
<01.10.2022 20:22:36> FTP Session 8 127.0.0.1 [Response] 220-Please visit https://filezilla-project.org/
<01.10.2022 20:22:36> FTP Session 8 127.0.0.1 [Response] 220 Welcome to xxxxxxxx FileZilla Server
<01.10.2022 20:22:36> FTP Session 8 127.0.0.1 [Command] AUTH TLS
<01.10.2022 20:22:36> FTP Session 8 127.0.0.1 [Response] 234 Using authentication type TLS.
<01.10.2022 20:22:36> FTP Session 8 127.0.0.1 [Trace] tls_layer_impl::server_handshake()
<01.10.2022 20:22:36> FTP Session 8 127.0.0.1 [Trace] tls_layer_impl::continue_handshake()
<01.10.2022 20:22:36> FTP Session 8 127.0.0.1 [Trace] tls_layer_impl::continue_handshake()
<01.10.2022 20:22:36> FTP Session 8 127.0.0.1 [Trace] tls_layer_impl::continue_handshake()
<01.10.2022 20:22:36> FTP Session 8 127.0.0.1 [Trace] tls_layer_impl::continue_handshake()
<01.10.2022 20:22:36> FTP Session 8 127.0.0.1 [Trace] TLS Handshake successful
<01.10.2022 20:22:36> FTP Session 8 127.0.0.1 [Trace] Protocol: TLS1.2, Key exchange: ECDHE-X25519-ECDSA-SHA256, Cipher: AES-256-GCM, MAC: AEAD, ALPN:
<01.10.2022 20:22:36> FTP Session 8 127.0.0.1 [Command] USER xxxx
<01.10.2022 20:22:36> FTP Session 8 127.0.0.1 [Response] 331 Please, specify the password.
<01.10.2022 20:22:36> FTP Session 8 127.0.0.1 [Command] PASS ****
<01.10.2022 20:22:36> FTP Session 8 127.0.0.1 mike [Response] 230 Login successful.
<01.10.2022 20:22:36> FTP Session 8 127.0.0.1 mike [Command] PBSZ 0
<01.10.2022 20:22:36> FTP Session 8 127.0.0.1 mike [Response] 200 PBSZ=0
<01.10.2022 20:22:36> FTP Session 8 127.0.0.1 mike [Command] PROT P
<01.10.2022 20:22:36> FTP Session 8 127.0.0.1 mike [Response] 200 Protection level set to P
<01.10.2022 20:22:36> FTP Session 8 127.0.0.1 mike [Command] FEAT
<01.10.2022 20:22:36> FTP Session 8 127.0.0.1 mike [Response] 211-Features:
<01.10.2022 20:22:36> FTP Session 8 127.0.0.1 mike [Response] 211 End
<01.10.2022 20:22:36> FTP Session 8 127.0.0.1 mike [Command] OPTS UTF8 ON
<01.10.2022 20:22:36> FTP Session 8 127.0.0.1 mike [Response] 202 UTF8 mode is always enabled. No need to send this command
<01.10.2022 20:22:36> FTP Session 8 127.0.0.1 mike [Command] SYST
<01.10.2022 20:22:36> FTP Session 8 127.0.0.1 mike [Response] 215 UNIX emulated by FileZilla.
<01.10.2022 20:22:36> FTP Session 8 127.0.0.1 mike [Command] SIZE /temp/test1.bin
<01.10.2022 20:22:36> FTP Session 8 127.0.0.1 mike [Response] 213 1000000000
<01.10.2022 20:22:36> FTP Session 8 127.0.0.1 mike [Command] TYPE I
<01.10.2022 20:22:36> FTP Session 8 127.0.0.1 mike [Response] 200 Type set to I
<01.10.2022 20:22:36> FTP Session 8 127.0.0.1 mike [Command] PASV
<01.10.2022 20:22:36> FTP Session 8 127.0.0.1 mike [Trace] Trying listen(1, 0) for data connection.
<01.10.2022 20:22:36> FTP Session 8 127.0.0.1 mike [Response] 227 Entering Passive Mode (127,0,0,1,211,243)
<01.10.2022 20:22:36> FTP Session 8 127.0.0.1 mike [Trace] session::on_socket_event(): source = data listen, flag = 2, error = 0, state = -1
<01.10.2022 20:22:36> FTP Session 8 127.0.0.1 mike [Trace] tls_layer_impl::server_handshake()
<01.10.2022 20:22:36> FTP Session 8 127.0.0.1 mike [Trace] tls_layer_impl::continue_handshake()
<01.10.2022 20:22:36> FTP Session 8 127.0.0.1 mike [Trace] tls_layer_impl::continue_handshake()
<01.10.2022 20:22:36> FTP Session 8 127.0.0.1 mike [Command] RETR /temp/test1.bin
<01.10.2022 20:22:36> FTP Session 8 127.0.0.1 mike [Response] 150 Starting data transfer.
<01.10.2022 20:22:36> FTP Session 8 127.0.0.1 mike [Trace] tls_layer_impl::continue_handshake()
<01.10.2022 20:22:36> FTP Session 8 127.0.0.1 mike [Trace] tls_layer_impl::continue_handshake()
<01.10.2022 20:22:36> FTP Session 8 127.0.0.1 mike [Trace] TLS Handshake successful
<01.10.2022 20:22:36> FTP Session 8 127.0.0.1 mike [Trace] TLS Session resumed
<01.10.2022 20:22:36> FTP Session 8 127.0.0.1 mike [Trace] Protocol: TLS1.2, Key exchange: ECDHE-X25519, Cipher: AES-256-GCM, MAC: AEAD, ALPN:
<01.10.2022 20:22:36> FTP Session 8 127.0.0.1 mike [Trace] session::on_socket_event(): source = data, flag = 2, error = 0, state = 2
<01.10.2022 20:22:41> FTP Session 8 127.0.0.1 mike [Trace] tls_layer_impl::shutdown()
<01.10.2022 20:22:41> FTP Session 8 127.0.0.1 mike [Trace] tls_layer_impl::continue_shutdown()
<01.10.2022 20:22:41> FTP Session 8 127.0.0.1 mike [Trace] data_socket_->shutdown() = 0
<01.10.2022 20:22:41> FTP Session 8 127.0.0.1 mike [Response] 226 Operation successful
So, now please help me reconcile this with the SSL Session Resume problem. I need to recreate this and cannot.
You wrote:
Later versions of FileZilla have removed the ability to not require TLS resumption and insist that it is absolutely necessary to create a secure connection
Then how did I get connected? What is different? Any ideas?
I do know by now that if you use TLS 1.3, you will have a problem, it seems. But with TLS 1.2, it should be ok.
@FanDjango It worked at first. We were able to connect and did a few initial (successful) tests uploading some files. Then it suddently started failing to connect and we haven't been able to reconnect since. We started on Filezilla 1.2 where the problem initiall appeared, then upgraded to 1.5.1 but the issue remained. I will check on what version of TLS we are using.
It worked at first. We were able to connect and did a few initial (successful) tests uploading some files. Then it suddently started failing to connect and we haven't been able to reconnect since.
Is this server actually under your control?
The FileZilla log should show you the error message that Filezilla would like to send to your client. Can you check there to see what it says?
This kind of thing always somehow sounds fishy.
If you use the new Nuget V41 released just today, the connect message will tell you the protocol that it is using.
Hi @FanDjango. I'm working on the same project as @MarkDynna.
Setting FluentFTP to connect with TLS 1.2 works, with one caveat. With TLS 1.3, the error message we were receiving from FileZilla was 425: TLS session of data connection not resumed. Though the files are being uploaded successfully now, we're still seeing an error with code 425 appear in the logs (though it may be a different error - the error message now is "425 Error while transfering data: ECONNABORTED - Connection aborted"), and the FtpResults returned from UploadDirectory() all have IsSuccess set to false. The FtpResults also have IsFailed set to false and no Exception, but we'd like a more conclusive return result.
It's worth noting that the 425 errors only occur when trying to upload a file, not while connecting to the server.
I have pored over this entire thread, but I can't find any reference to the FluentFTP version you are using, please forgive me if I am blind.
Depending on your version of FluentFTP: There have been some changes towards how sessions are terminated more or less recently. Probably you are seeing FileZilla complaining about an unclean shutdown of the encrypted session.. stuff like that has been fixed in the last few months.
See https://github.com/robinrodricks/FluentFTP/issues/771 and https://github.com/robinrodricks/FluentFTP/issues/930.
If the bug still persists, more information from you might have been useful.
Maybe if you had attached the log I might have seen something familiar. I don't like debugging with a crystal ball :-)
One more piece of information:
Currently, we no longer support TLS 1.3, especially in combination with SSL Session Resume. We are searching for ways to get it work reliably but it does not look good at the moment.
SSL Session Resume works with TLS 1.2. The error you now describe is a different one, quite sure of that.
The OP @BrianCryer is long gone, it seems. So let's do the following and continue here with the new title.
Hi @FanDjango. I believe we're on version 38.0.0. We got the latest version sometime in August. I've attached a log where I uploaded a directory of three files successfully, then re-uploaded the files to overwrite them. As before, the files were uploaded successfully but the FtpResults were all set to IsSuccess = False.
Thanks for your help.
Have you got
GnuTLS error -110 in gnutls_record_recv: The TLS connection was non-properly terminated.
Client did not properly shut down TLS connection
425 Error while transfering data: ECONNABORTED - Connection aborted
in FileZilla?
I believe we're on version 38.0.0
To what extent would it be possible for you to use V41?
I see those exact messages in the FileZilla logs. We can certainly upgrade to V41 if those issues have been fixed since V38. Thanks again.
Please note that we still have one issue open for this, which we still cannot explain. I am standing by for your report.
Here is some research on this:
Nearly a month ago PR https://github.com/robinrodricks/FluentFTP/pull/944 addressed exactly this problem. Merged into master on September 10.
It came into Nuget Release V41.0.0 on October 2.
https://github.com/robinrodricks/FluentFTP/pull/944 explicitely mentions the wrong order of ssl stream shutdown, as seen from a vsftp server. This happened on any server type - and each server type reacts differently to this. Some more lenient and some less so.
If you read https://github.com/robinrodricks/FluentFTP/pull/944 and also read the article mentioned therein (this, you will agree it is the same problem as yours.
Ergo - I advise updating to FluentFTP V41.
We have the latest code as of yesterday, but we're still running into the same issues. I've attached a client log for an upload which shows the same "connection aborted" messages as before. Those three lines you mentioned the other day are still in the FileZilla log as well.
I am overjoyed. I have reproduced this. I am moving this to a new issue, debug to try and find a fix and will invite you to participate there. It was the COMPLETE log that allowed me to recreate the exact situation. Once again I must plead for maximum verbose complete logs etc. instead of piecemeal information that requires so much back-and-forth.
Since there are links to this issue in the wider internet out there, where the original problem reported was the failure to "Session Resume", I have re-added the corresponding tag. The second half of this issue somehow "migrated" to the other big issue (at that time): Namely "Incorrect SSL shutdown".
So this closed issue actually contains two separate problems.
I am overjoyed. I have reproduced this. I am moving this to a new issue, debug to try and find a fix and will invite you to participate there. It was the COMPLETE log that allowed me to recreate the exact situation. Once again I must plead for maximum verbose complete logs etc. instead of piecemeal information that requires so much back-and-forth.
Great work!!