piskvorky
ssh/scp/sftp: enable pipelining of write operations
Turn on pipelining of write operations for ssh/scp/sftp
Motivation
It significantly speeds up writes. From paramiko docs:
When pipelining is on, paramiko won’t wait for the server response after each write operation. Instead, they’re collected as they come in. At the first non-write operation (including close), all remaining server responses are collected. This means that if there was an error with one of your later writes, an exception might be thrown from within close instead of write.
Popular sftp client library pysftp enables pipelining for put operations.
Result of simple benchmark
pipelining disabled:
Total time: 33.55 seconds, sent: 2621440 bytes, speed: 76 kbytes/sec
pipelining enabled:
Total time: 3.58 seconds, sent: 2621440 bytes, speed: 714 kbytes/sec
Thank you for your contribution. Sorry it has taken so long for me to review this.
Please have a look at the comments and get back to me.
@mrk-its Ping. Are you able to finish this PR?
Yes, sure, I'll get back soon with changes.