Feature: Ability to login as one user and run all tasks as another

[theozaurus]

We've created a backend that allows Capistrano 3 to work with setups that require users to login in as user A, but run all commands as user B.

The backend essentially sets a global owner and all tasks (unless explicitly overridden) will run as this user. There are a few other options that can also be globally configured (this helps get around certain use cases) such as directory and the ability to forward an SSH agent to the owner for certain commands. This means the existing Capistrano tasks do not need to be reimplemented (instead the backend is switched).

While this kind of server setup is not recommended or ideal, when coming across one it's still useful to be able to use Capistrano to deploy.

The code is pretty ugly at the moment, subclassing and overriding a lot of private methods. However, if core sshkit are interested this could easily be cleaned up. The test suite style and methodology has been retained to try and make it as easy as possible to merge into core sshkit.

Does this look like a feature that you would be interested in merging into core? If so I can raise a PR.

[leehambley]

Thanks @theozaurus that sounds really interesting, I'll have to review more thoroughly, but I'd really suggest that if you would be fine maintaining it, we could consider it as a "2nd party' backend, we could host you under capistrano/ on Github, but give you control of everything, and mention that it's a "power backend" in the docs, and refer people to it.

I'm in favour of keeping it external, and if there's work we need to do in Capistrano to make backends more pluggable, I would support you in that.

[robd]

Hey

I would like to see support for this added by supporting default command options in the command map. I plan to implement something like this to enable interactive sudo support as discussed here: https://github.com/capistrano/sshkit/pull/234#issuecomment-99062791

Something like this:

SSHKit.config.command_map.default_options[:sudo] = {
  as: 'some_user'
}

Would that work for you?

I plan to do this once master is released, because I don't think we should embark on another big change in this release.

[leehambley]

I might rather prefer that we put that in new "default command options" map, but yeah, that seems reasonable.

[theozaurus]

@robd and @leehambley that kind of setup sounds great. In which case we can leave this backend hosted on the FundingCircle org, and deprecate it when the "default command options" functionality is added.

[robd]

@theozaurus OK that sounds good. I will get back in contact once I come to work on this, but I'm a bit tied up with client work at the moment, so I don't know when that will be. I will also look in more detail at the backend work you've done - especially user support file upload which looks interesting. Cheers!

[theozaurus]

@robd No problem. The command mapping should do most of it. A little bit of it can probably be moved into the netssh backend (file upload in an as block for example).

[marcovtwout]

@theozaurus I have exactly this use case (login as a, run everything as b). Is there a way to accomplish this through the default command options yet?

[theozaurus]

@marcovtwout There is currently no way (AFAIK) to do this using default command options yet. However, give the sshkit-backends-netssh_global a go. It worked very effectively for when we needed it.

[lizgene]

@theozaurus Any update on this request? We have this exact use case going on.

[theozaurus]

@lizhubertz Are you able to use the backend here: https://github.com/fundingcircle/sshkit-backends-netssh_global ? I've moved on from Funding Circle so have no idea if they are still using it or whether it is compatible with the latest version of Capistrano. There are a bunch of tests we modified to demonstrate how it works which should help if you need to modify anything.

[lizgene]

@theozaurus I am, still struggling to get it to work though. I'll check out the tests! If there's anything in particular you can point me to, that would be super helpful.

[theozaurus]

@lizhubertz is it spitting out an error message of any sort? It's been so long I'm not sure I can give any pointers.

[lizgene]

@theozaurus Thanks! So background, I set up a simple test task to try and see how this is working.

desc "Check that we can access everything"
task :check_permissions do
  on roles(:all) do |host|
    execute("whoami")
    execute("cp /tmp/file-owned-by-deploy-user /tmp/test-file")
  end
end

My situation is: Because of security configurations that I cannot change, I need to ssh onto all of my boxes a a federated user, myuser but everything on my box that I need to deploy to is owned by deployuser and located within /home/deployuser. myuser can assume user deployuser and has all sudo privileges.

Ultimately, I want my deployments to function as if I ssh'd in as deployuser from the get-go, so every single command is run as deployuser and from the deployuser's home directory.

The first thing I tried was from the sshkit-backends-netssh_global documentation. I added this to my Capfile:

require 'sshkit/backends/netssh_global'

SSHKit::Backend::NetsshGlobal.configure do |config|
  config.owner        = 'deployuser'
  config.directory    = '/home/deployuser'
end

This doesn't seem to have any effect whatsoever. No new error messages, just nothing. Permission denied, and whoami returns myuser.

Second, I tried to set :ssh_backend from Capistrano, which I saw in this issue for sshkit-backends-netssh_global:

require 'sshkit/backends/netssh_global'

set :sshkit_backend, -> {
  SSHKit::Backend::NetsshGlobal.tap do |backend|
    backend.configure do |config|
      config.owner        = 'deployuser'
      config.directory    = '/home/deployuser'
    end
  end
}

This does something but hits an error with a deprecated method checkout

NoMethodError: undefined method `checkout' for #<SSHKit::Backend::ConnectionPool:0x007fae8340e3c0>

The method still exists here

I kinda feel like I'm going crazy here though because, when I try to wrap my task in as 'user' as per the SSHKit documentation, whoami gets run as deployuser while the cp command is not. So maybe my test script isn't doing what I think it's doing.

desc "Check that we can access everything"
task :check_permissions do
  on roles(:all) do
    as 'deployuser' do
      execute("whoami") # this is deployuser

      # this throws permission denied (but can be copy/pasted onto the server and run as deployuser, so wtf)
      execute("cp /tmp/file-owned-by-deploy-user /tmp/test-file")
    end
  end
end

Another (similar) tactic I tried, which results in the same as the above example is:

SSHKit.config.command_map = Hash.new do |hash, command|
  hash[command] = "sudo -u deployuser #{command}"
end

One problem being, even though this works to append sudo -u deployuser on commands, all these commands are still being run from /home/myuser. I was hoping that SSHKit::Backend::NetsshGlobal would help me solve that problem with config.directory

Also, my bundle includes the latest of all of the things:

capistrano (3.10.1)
sshkit-backends-netssh_global (0.1.1)

One thing I'm wondering is that maybe sshkit-backends-netssh_global (0.1.1) used to work, then broke with a later version of capistrano. In which case, I'd be totally willing to downgrade.

I also opened a StackOverflow question

[theozaurus]

@lizhubertz Thanks for all the details.

I've taken a look over the Funding Circle code and have updated a fork of it (previously it only supported up to 1.8.x (https://github.com/capistrano/sshkit/commit/6de86147c015cea2c7f851a2e9e65fc91d47b9ee was one of the breaking API changes).

Can you try out: https://github.com/theozaurus/sshkit-backends-netssh_global and let me know how you get on?

[lizgene]

@theozaurus Awesomeness. I just bundled and removed my monkey patches, and I'm getting somewhere. My whoami task returns my deployuser, which is great.

One issue I just ran into:

KeyError: key not found: :user
/Users/hubertz/.rbenv/versions/2.3.4/lib/ruby/gems/2.3.0/bundler/gems/sshkit-backends-netssh_global-3d1ceb022ccd/lib/sshkit/backends/netssh_global.rb:40:in `fetch'
/Users/hubertz/.rbenv/versions/2.3.4/lib/ruby/gems/2.3.0/bundler/gems/sshkit-backends-netssh_global-3d1ceb022ccd/lib/sshkit/backends/netssh_global.rb:40:in `ssh_user'
/Users/hubertz/.rbenv/versions/2.3.4/lib/ruby/gems/2.3.0/bundler/gems/sshkit-backends-netssh_global-3d1ceb022ccd/lib/sshkit/backends/netssh_global.rb:24:in `upload!'
/Users/hubertz/.rbenv/versions/2.3.4/lib/ruby/gems/2.3.0/gems/capistrano-3.10.1/lib/capistrano/scm/tasks/git.rake:9:in `block (3 levels) in eval_rakefile'
/Users/hubertz/.rbenv/versions/2.3.4/lib/ruby/gems/2.3.0/gems/sshkit-1.15.1/lib/sshkit/backends/abstract.rb:29:in `instance_exec'
/Users/hubertz/.rbenv/versions/2.3.4/lib/ruby/gems/2.3.0/gems/sshkit-1.15.1/lib/sshkit/backends/abstract.rb:29:in `run'
/Users/hubertz/.rbenv/versions/2.3.4/lib/ruby/gems/2.3.0/gems/sshkit-1.15.1/lib/sshkit/runners/parallel.rb:12:in `block (2 levels) in execute'

Here's the offending line

Have you seen this?

I added set :user, 'sshuser' above where I declare set :sshkit_backend, SSHKit::Backend::NetsshGlobal to no avail. Not sure why it's not grabbing things as expected.

I'm thinking maybe I need to set it as ssh_options reading these lines

Update

That was it. I just had to:

set :ssh_options, {
  user: 'sshuser'
}

[theozaurus]

You should be able to do:

SSHKit::Backend::NetsshGlobal.configure do |config|
  ...
  config.ssh_options = { user: 'sshuser' }
  ...
end

However, it's weird you have to do that. How are your servers defined? Do they have a user set? e.g.:

server "example.com", user: 'sshuser', roles: ['app']

[theozaurus]

@lizhubertz does the system work okay now? If it does I'll raise a PR to Funding Circle to get those changes in and a new gem released.

[lizgene]

@theozaurus I wanted to wait until I got it working to answer back - but short answer is yes, with some monkey patches!

The 3 monkey patches I had to use were:

rake:assets:precompile

Within v. capistrano-rails 1.3.1, the deploy:assets:precompile task doesn't respect the deploy user or RAILS_ENV. I also decided to add my npm install here, since I needed to monkey patch it anyway. I don't know if this has anything to do with your code, but recording it for posterity.

Rake::Task["deploy:assets:precompile"].clear
namespace :deploy do
  namespace :assets do

    desc 'Monkey patch: Execute rake assets:precompile within the correct RAILS_ENV'
    task :precompile do
      on release_roles(fetch(:assets_roles)) do
        execute "cd #{release_path} && sudo -u deployuser HOME=/home/deployuser npm install --production --silent && sudo -u deployuser RAILS_ENV=#{fetch(:rails_env)} my-bundle exec rake assets:precompile"
      end
    end
  end
end

deploy:migrating & deploy:cleanup

The second and third monkey patches were in capistrano 3.10.1 proper.

For whatever reason, deploy:migrating wasn't respecting RAILS_ENV either, plus wasn't sudo'ing as the deployuser.

namespace :deploy do
  desc "Monkey patch: Migrate the db"
  task :migrating do
    on roles(:db_role) do
      execute "cd #{release_path} && sudo -u deployuser RAILS_ENV=#{fetch(:rails_env)}_migrate #{fetch(:rake)} db:migrate"
    end
  end
end

And finally, in deploy:cleanup, theres an execute rm statement that requires the insertion of sudo rm. I think the way they wrote it in the Capistrano script itself means that it does need to be monkey patched or fixed in Capistrano.

other weirdness

The only other point of weirdness I came across was in your use of setfacl. I put acl on our servers, but the command was attempting to call setfacl on the user defined in config.ssh_options = { user: 'sshuser' }

02 setfacl -m u:sshuser:rwx /tmp; true
02 setfacl: /tmp: Operation not permitted

This WOULD be fine, except that the way our ssh is configured, I use my_personal_name to initially ssh onto the box, which runs a script to check for my keys, and then officially logs me in as federateduser. So setfacl needs to run with federateduser, but since it's pulling in sshuser, that command fails (because sshuser doesn't actually exist on the box). This is something kind of weird to our setup, but it would be nice to not have that dependency that the user that you ssh with is automatically the user you want to have when running the setfacl command.

I'll probably end up monkey patching this one as well, just to get rid of the error messages. It wasn't necessary to run this command, but it does muddy up the output.

[theozaurus]

Thanks for the update. I've made a pull request as it looks like it is largely working

With reguard to your issues:

1. I'm not sure why the deploy user or RAILS_ENV is not being respected - it's hard to tell where that is going wrong
2. Similar to above, hard to rule out a fault in the backend
3. The backend does a lot of setfacl. The reason for it is because when a file is uploaded it is first uploaded as the SSH user, then setfacl has to be used to set it to the federateduser. It's very strange that setfacl is being done as sshuser. There are tests relating to that functionality here: https://github.com/theozaurus/sshkit-backends-netssh_global/blob/master/test/functional/backends/test_netssh_global.rb#L230

[mattbrictson]

@theozaurus @lizhubertz apologies for interrupting the flow of this discussion, but could you move future comments on this topic to the https://github.com/theozaurus/sshkit-backends-netssh_global repo? That seems like a more appropriate permanent home for troubleshooting issues regarding that gem. Thanks!

[stuarthannig]

+1 As security becomes more focussed on server setup, this seems very worthy.