sshfs-win
sshfs-win copied to clipboard
winfsp
Accessing files is very very slow
Hello, I installed sshfs-win and connected to an SFTP location. It is very slow when i open a 28MB text file. Takes about 4 minutes. Same file is downloaded with Filezilla in 8 seconds.
That is not normal and I do not have a good explanation for you. What is the operation that takes 4 minutes to complete?
@billziss-gh I've done some throughput testing with mpv, a media player, in mpv-player/mpv#6802 and come to the conclusion that sshfs-win or WinFsp handles small reads really badly.
With mpv's default read size of 2 KiB sshfs-win delivers a mere 30 KiB/s. Increasing read sizes to 1 MiB it can achieve up to 100x that.
At the very bottom I've done tests on the local network and even then sshfs-win can't saturate 100mbps. Samba doesn't care about read sizes at all and always saturates gigabit network.
@Marco01809
Please note that sshfs-win in its default configuration does not use the file data cache.
If you launch sshfs from the command line and use the -o FileInfoTimeout=-1 option then the file data cache should be enabled and you should get vastly better read speeds and read ahead from the OS cache manager.
@billziss-gh Can't really get that to work. Took me quite a while to figure out you're supposed to paste the password into the terminal, but I cant get past that:
C:\Program Files\SSHFS-Win\bin>sshfs-win.exe svc \\sshfs\[email protected]!1234\storage S: WORKGROUP+MyWindowsUser -o FileInfoTimeout=-1
<pasted my password here>
OK
Cannot create WinFsp-FUSE file system.
The service sshfs has failed to start (Status=c000000d).
Not sure what to put for LOCUSER, I also tried to use just the windows account name or an empty string.
The same prefix works fine when I use the Map drive functionality in Explorer.
Try with only one backslash before the sshfs part. Like this:
sshfs-win.exe svc \sshfs\[email protected]!1234\storage S: WORKGROUP+MyWindowsUser -o FileInfoTimeout=-1
I have the same issue and using the suggested option does not fix it.
For instance, opening a simple 2kb text file in notepad takes roughly 3 seconds. And the Gbit Network is not a problem here. Copy larger files is also quick and not a problem. It really is just very small files.
Doing some performance testing I found a extremely interesting detail. Network traffic send is double of what the written total file sizes are. (so disk write speed 30MB/s and server network send speed of 60MB/s) Tested for random content file sizes from 1k to 1M. Could there be a bug resulting in all the data being requested and processed twice? That would probably mess with windows kernel stuff resulting in huge performance hits with small files?
The suggested FileInfoTimeout=-1 option didn't change anything for me either
@OskarAtGitHub This sounds familiar, in some tests on local gigabit network I did a while ago the throughput was pretty obviously "capped" at ~52 MiB/s, which would be roughly half of gigabit (minus some overhead). But I can't reproduce it anymore, now I get between 70-80MiB/s write speed at 82-92MiB/s network speed (might be a CPU bottleneck on the server, didn't check).
From my testing today this 2:1 ratio holds for all file sizes from 1k to 1M. (With 1k files, I was getting a throughput of about 1MB/s, with a network rate of 2MB/s.)
@OskarAtGitHub This sounds familiar, in some tests on local gigabit network I did a while ago the throughput was pretty obviously "capped" at ~52 MiB/s, which would be roughly half of gigabit (minus some overhead). But I can't reproduce it anymore, now I get between 70-80MiB/s write speed at 82-92MiB/s network speed (might be a CPU bottleneck on the server, didn't check).
@OskarAtGitHub I am uncertain why you would see half the write speed from the network send speed. I doubt that this is related to the file system layer and it may have more to do with how sshfs does network communication.
@Marco01809 The FileInfoTimeout=-1 option would only be useful if you do multiple/random I/O on a file (e.g. if you are working with it, doing edits, saving it, etc.) The reason is that you would be working against the OS cache rather than hitting the network for all this I/O.
If you are simply doing speed tests copying a file sequentially, FileInfoTimeout=-1 will be less useful.
@billziss-gh This was a interesting question. I just did comparison tests on my example randomly generated files. I have separate directories with 1k, 10k, 100k and 1000k size files. sshfs-win copy (admittedly with windows explorer on win 10) results in a 2:1 ratio of network data to transferred file size. Using scp between linux machines results in a 1:1 ratio for the same data. This behavior is the same for all tested file sizes. I'm not sure if it's related or not, but if the commands are being issued twice, that might explain why some programs are unable to save files to the sshfs-win -mounted drives. Firefox for instance saves temp files during download, then does some copy-delete or similar when done. End result is the saved file being deleted. (from firefox saves I can see the temp files and delete stuff happening on the server with disk activity monitoring the directory, not that good at using it though)
Tested with xcopy on win 10. Result is much closer to 1:1. Difficult for me to get a accurate number, but maybe 1.1:1 or so. So the 2:1 is clearly a windows explorer related thing. Very strange!
I would not be surprised if Explorer re-reads the file right after writing it for verification purposes. In this case the FileInfoTimeout==-1 should help, unless Explorer opens files for copying with caching disabled.
I am having incredible slowness with WinFsp and rclone. I reported the problem on the rclone forums.
It takes WinFsp 17 seconds to list items in a directory via rclone mount, whereas listing the items directly (via S3-compatible APIs) takes about 1.2 seconds.
