FluentFTP icon indicating copy to clipboard operation
FluentFTP copied to clipboard

DownloadDirectoryAsync issues, pausing download after every file

Open heavyflames opened this issue 3 years ago • 6 comments

FTP OS: Unix / Windows / Embedded

FTP Server: Pure-FTPd / DrFTPD / Vsftpd / ProFTPD / Vax / VMS / OpenVMS / Tandem / HP NonStop Guardian / IBM OS400 / AS400 / Windows CE

Computer OS: Windows 2008 R2 Server

FluentFTP Version: 33.0.3

When trying to download a directory and all it files, theres a delay between every file it seems like.

Im using a progress to log the progress of each file

Progress<FtpProgress> progress = new Progress<FtpProgress>(p => {
                    if (p.Progress == 1)
                    {
                        // all done!
                    }
                    else
                    {
                        // percent done = (p.Progress * 100)
                        Console.WriteLine("File " + p.LocalPath +  " PROGRESS : " + p.Progress * 100 + " Speed : " + p.TransferSpeedToString());
                    }
                });

                //DownloadFile(localFilePath, _ip, _username, _password, path);
                await conn.DownloadDirectoryAsync(localFilePath, path, FtpFolderSyncMode.Update,FtpLocalExists.Append, FtpVerify.None,null, progress);

Im using these settings:

conn.SslProtocols = System.Security.Authentication.SslProtocols.None;
                conn.EncryptionMode = FtpEncryptionMode.Explicit;
                conn.ValidateAnyCertificate = true;
                conn.RetryAttempts = 5;
                conn.ConnectTimeout = 30000;
                conn.ReadTimeout = 30000;
                conn.DataConnectionConnectTimeout = 30000;
                conn.DataConnectionReadTimeout = 30000;
               // conn.SocketPollInterval = 15000;
                //conn.SocketKeepAlive = true;
                conn.DownloadRateLimit = 0;
                conn.TransferChunkSize = 1024 * 256;
                conn.LocalFileBufferSize = 1024 * 256;

Any idea why this is happening?

It seems to be 30 seconds between every "pause"

heavyflames avatar Mar 05 '21 09:03 heavyflames

Ok so its not a pause between every file, it downloads a couple of files at the time it seems like, then pauses for 30s...

heavyflames avatar Mar 05 '21 09:03 heavyflames

I have tried to play a little with the settings, setting all the timeouts to 5 seconds, and using SocketPollInterval = 0 and SocketKeepAlive = true. It now downloads files in a "better" intervall, but still the number of files that are donwloaded varies alot, sometimes 1 file is downloaded and with a pause of 5-20 seconds, and sometimes it downloads 5-10 files with a pause of 5-20 seconds...I dont get why.. The file sizes are almost the same, they are around 150-200kb each.

heavyflames avatar Mar 05 '21 12:03 heavyflames

Console.WriteLine will be causing a delay. I'm not sure why else there should be a delay - all code is direct and we don't use any third party async/threading code. Also, the only time we delay intentionally is for rate limiting, which looks like you have disabled.

robinrodricks avatar Mar 09 '21 05:03 robinrodricks

What you can do is take the source code of FluentFTP and build from source and add that reference to your project, as if its one of your own projects. Now when the delay happens, press PAUSE in your debugging tools in VS. This should take you right to the line that is pausing the system.

robinrodricks avatar Mar 09 '21 05:03 robinrodricks

Hi, the thing is that it goes very fast when debugging on my computer using a ftp server localy on my machine. But when using the code on the server, that is using a FTP on another server(they are still on the same network), it is very slow. When using filezilla on the server and downloading the files that way, its very fast...

heavyflames avatar Mar 15 '21 15:03 heavyflames

Filezilla is C++, FluentFTP is .NET. Obviously it will be faster.

robinrodricks avatar Mar 21 '21 06:03 robinrodricks

@heavyflames Is this issue still open for you?

The delay could be the connection being dropped mid transfer. To really diagnose this well, a full verbose log would be needed, also to get the best debugging messages in the log, using the newest Fluentftp version.

FanDjango avatar Nov 04 '22 20:11 FanDjango