Allow to choose between sftp and scp.

[leehambley]

Because well, it simply makes sense.

[besi]

+1

@leehambley Which one is the current default? My hosting provider only supports SFTP so I am curious if SSHKit is an option for me at all.

[leehambley]

https://github.com/capistrano/sshkit/search?utf8=%E2%9C%93&q=SCP

[ideaoforder]

+1

[uliunger]

+1

[aleksejkuzmin]

+1

[mlyubarskyy]

+1

[dc519]

+1

[drewbug]

:+1:

[JasonPoll]

It would be fantastic if this issue could be revisited.

These days, modern CLI SCP clients use the SFTP protocol by default. Net::SCP doesn't implement this "silently using SFTP protocol under the hood" tactic -- it's intentionally an implementation of the SCP protocol and only the SCP protocol.

Further, with major distributions like RHEL disabling the SCP protocol by default (as of RHEL9,) and with promise to remove support entirely in the future, SSHKit's reliance on Net::SCP with no option to use Net::SFTP instead of Net::SCP is (IMO) a big problem.

[mattbrictson]

@JasonPoll thanks for resurfacing this. I am curious if you have any suggestions about how to we could make the transition from SCP to SFTP.

Implementing SFTP is easy, but I worry about managing the migration for the many existing users of sshkit/capistrano/etc. that currently rely on SCP.

Would we allow users to configure SFTP vs SCP? Would it be a global option or per host? Or is support for SFTP ubiquitous enough that we could drop SCP altogether? I just don't want to break existing workflows if possible.

One possibility is to introduce a new :transfer_protocol option in ssh_options. Individual hosts already support arbitrary ssh_options, and they can also be specified globally via SSHKit::Backend::Netssh.configure, so this seems like a natural place for it. The :transfer_protocol would allow :scp and :sftp as allowed values. Then in sshkit's upload! and download! implementations, we consult this option to decide what protocol to use.

Initially we could default to :scp to maintain backwards compatibility, while allowing users to opt into :sftp. Then in a subsequent release we could switch the default to :sftp if we want to.

Thoughts?

[JasonPoll]

@mattbrictson - agreed that managing the migration is an important consideration. Avoiding a breaking change is preferred, especially when SCP is (and will likely continue to be) a viable option for many.

For context, my primary usage of SSHKit is through Capistrano and may thoughts my miss important details pertinent to other SSHKit use cases.

Honestly, everything you've suggested sounds like an excellent plan.

• Opt-in for SFTP since SCP likely has use cases, legacy or otherwise.
• Global or per-host configuration.
• Default the new :transfer_protocol to :scp to avoid a potentially-breaking change upon initial release.

I think the only things I'd add to your ideas:

• Documentation which clearly describes how to use your proposed :transfer_protocol, with global and per-host examples.
• If it's decided that support for SCP should be phased out entirely, implementing Gem::Deprecate in some way that denotes SCP support will be removed in a future version.
• Are there any downsides or concerns with adding Net::SFTP as a dependency to SSHKit? Net::SFTP's readme notes that the project is in maintenance mode. 🤷
  • I've briefly looked at your Tomo -- I don't want to scope-creep this, but is there any merit in SSHKit removing dependencies on external gems for SSH/SCP/SFTP functionality in the future?
• Are you aware of any follow-up development that would need to happen on Capistrano? I've only had to lightly utilize ssh_options in my Capistrano configurations through the years, and have no current usage of it -- I don't want to miss something important.

[mattbrictson]

@JasonPoll I think ssh_options will pass through to SSHKit without any changes on the Capistrano side, so we should be good there.

You're right that net-sftp would have to be added as a new gem dependency. I think that's OK, given that the only alternative would be to rebuild the foundation of SSHKit to use a different backend implementation. I tried that with my Tomo project (i.e. using the native ssh binary instead of net-ssh), and it works great, but I think it would be a huge undertaking and go against the pure Ruby nature of Capistrano/SSHKit's design.

Based on your feedback it seems that we are mostly on the same page with the approach, so I think the next step will be for me to implement a proof-of-concept in the next couple weeks.

Would you be able to test my branch in a real-world scenario once I reach that point?

[JasonPoll]

Would you be able to test my branch in a real-world scenario once I reach that point?

@mattbrictson I can and will -- I'm good with Vagrant and Ansible and can spin up testing VMs that would closely mimic my production environments that disable SCP by default.