Gem will break after Sidekiq 6.0 has been released

[Flauschbaellchen]

After updating sidekiq to version 5.2.5 the following warning is shown:

WARNING: PID file creation will be removed in Sidekiq 6.0, see #4045. Please use a proper process supervisor to start and manage your services
WARNING: Logfile redirection will be removed in Sidekiq 6.0, see #4045. Sidekiq will only log to STDOUT
WARNING: Daemonization mode will be removed in Sidekiq 6.0, see #4045. Please use a proper process supervisor to start and manage your services

The ticket: mperham/sidekiq#4045

As the gemspec denotes a requirement to sidekiq >= 3.4 but no upper limit, this gem will most likely break after the new version of sidekiq has been released.

The quickfix would be to additionally require sidekiq <= 6.0 and release a new version. What needs to be done to fix this issue in the long run?

PS: I already had gem 'sidekiq', '~> 5.1' within my gemfile so I'm not directly affected but will prevent the update to any newer version of sidekiq.

[lufutu]

Any update or solution ?

[vilusa]

Hi everyone, I am using sidekiq 5.2.5 version

Error: WARNING: Daemonization mode will be removed in Sidekiq 6.0, see #4045. Please use a proper process supervisor to start and manage your services WARNING: PID file creation will be removed in Sidekiq 6.0, see #4045. Please use a proper process supervisor to start and manage your services

This errors puts console, Is there any solution or resource for this problem ?

[Paprikas]

For those who use rbenv like me, I did quick manual on how to switch to systemd. https://gist.github.com/Paprikas/11fc850f81b687d9cbb7a8efb5ead208 You have to remove old monitoring\ running scripts to avoid conflict.

[JoeDupuis]

I think adding a sidekiq <= 6.0 restriction on the last version is a bad idea. The right thing to do would be what the creator of sidekiq advocate. https://www.mikeperham.com/2014/09/22/dont-daemonize-your-daemons/ Deprecating the daemonization and force using a service manager.

capistrano-sidekiq already works with sidekiq 6 if you use the systemd init feature.

If somebody want to keep deamonizing sidekiq they can still force the version of sidekiq and capistrano-sidekiq in their gemfile. If you lock the sidekiq version in the gemspec, you force people who do the right thing to fork capistrano-sidekiq if they want the latest version of sidekiq. It's a lot easier and faster to lock the sidekiq version in your gemfile than it is to fork a gem.

Though, forcing systemd restrict deployment to linux servers. One way to deal with this would be to abstract the call to systemd in a class that we could substitute for other class if we want to use different service manager. Then adding a new service manager would be trivial.

[leoarnold]

For systemd-based deployments, it's basically:

• Tailor systemd unit template to your needs: https://github.com/mperham/sidekiq/blob/master/examples/systemd/sidekiq.service
• Extract Sidekiq log file from syslog if needed: https://stackoverflow.com/a/43830129
• Replace this gem with something like the following for user space systemd units:

set :sidekiq_roles, -> { :app }
set :sidekiq_systemd_unit_name, "sidekiq@#{fetch(:stage)}"

namespace :sidekiq do
  desc 'Quiet sidekiq (stop fetching new tasks from Redis)'
  task :quiet do
    on roles fetch(:sidekiq_roles) do
      # See: https://github.com/mperham/sidekiq/wiki/Signals#tstp
      execute :systemctl, '--user', 'kill', '-s', 'SIGTSTP', fetch(:sidekiq_systemd_unit_name), raise_on_non_zero_exit: false
    end
  end

  desc 'Stop sidekiq (graceful shutdown within timeout, put unfinished tasks back to Redis)'
  task :stop do
    on roles fetch(:sidekiq_roles) do
      # See: https://github.com/mperham/sidekiq/wiki/Signals#tstp
      execute :systemctl, '--user', 'kill', '-s', 'SIGTERM', fetch(:sidekiq_systemd_unit_name), raise_on_non_zero_exit: false
    end
  end

  desc 'Start sidekiq'
  task :start do
    on roles fetch(:sidekiq_roles) do
      execute :systemctl, '--user', 'start', fetch(:sidekiq_systemd_unit_name)
    end
  end

  desc 'Restart sidekiq'
  task :restart do
    on roles fetch(:sidekiq_roles) do
      execute :systemctl, '--user', 'restart', fetch(:sidekiq_systemd_unit_name)
    end
  end
end

after 'deploy:starting', 'sidekiq:quiet'
after 'deploy:updated', 'sidekiq:stop'
after 'deploy:published', 'sidekiq:start'
after 'deploy:failed', 'sidekiq:restart'

[lucascaton]

Sidekiq 6 has been released and invalid option: --index error messages are now showing up during deploys:

05:40 sidekiq:start
      01 /usr/local/rvm/bin/rvm default do bundle exec sidekiq --index 0 --pidfile /app/shared/tmp/pids/sidekiq-0.pid --environment production --logfile /app/shared/log/sidekiq.log …
      01 invalid option: --index
      01 /app/shared/bundle/ruby/2.6.0/gems/sidekiq-6.0.0/lib/sidekiq/cli.rb:345:in `parse_options'
      01 /app/shared/bundle/ruby/2.6.0/gems/sidekiq-6.0.0/lib/sidekiq/cli.rb:201:in `setup_options'
      01 /app/shared/bundle/ruby/2.6.0/gems/sidekiq-6.0.0/lib/sidekiq/cli.rb:24:in `parse'
      01 /app/shared/bundle/ruby/2.6.0/gems/sidekiq-6.0.0/bin/sidekiq:11:in `<top (required)>'
      01 /app/shared/bundle/ruby/2.6.0/bin/sidekiq:23:in `load'
      01 /app/shared/bundle/ruby/2.6.0/bin/sidekiq:23:in `<main>'
      01 /app/shared/bundle/ruby/2.6.0/bin/ruby_executable_hooks:24:in `eval'
      01 /app/shared/bundle/ruby/2.6.0/bin/ruby_executable_hooks:24:in `<main>'
#<Thread:0x00007fd9f96195b0@~/.rvm/gems/ruby-2.6.4/gems/sshkit-1.20.0/lib/sshkit/runners/parallel.rb:10 run> terminated with exception (report_on_exception is true):
Traceback (most recent call last):
	19: from ~/.rvm/gems/ruby-2.6.4/gems/sshkit-1.20.0/lib/sshkit/runners/parallel.rb:12:in `block (2 levels) in execute'
	18: from ~/.rvm/gems/ruby-2.6.4/gems/sshkit-1.20.0/lib/sshkit/backends/abstract.rb:31:in `run'
	17: from ~/.rvm/gems/ruby-2.6.4/gems/sshkit-1.20.0/lib/sshkit/backends/abstract.rb:31:in `instance_exec'
	16: # etc...

[ragesoss]

Thanks @leoarnold! Your example code was extremely useful.

[JoeDupuis]

This gem does support user level systemd deployment already. With customizable service template included.

You just have to set the correct parameters.

https://github.com/seuros/capistrano-sidekiq/blob/master/README.md#integration-with-systemd

[leoarnold]

@twistedjoe True that, yet most users do not require the versatile support for the whole variety of init systems. Since this project is not very active (see screenshot below), it seems advisable (c.f. CVE-2019-15224) to replace it with four simple and maintainable customized Capistrano tasks rather than hoping for a fresh breeze in this project.

[seuros]

I'm going to release the new version this weekend, that support only sidekiq 6+. As @twistedjoe noted, the current workaround is to use the upstart/systemd integration that is optional for now. In the next version, it will be the only way to do it.

[jules-w2]

Hey @seuros,

Do you have an ETA for this release? (sidekiq 6+) Thanks :)

[tcd]

@seuros ping

[rusikf]

+1, I have

 /gems/sshkit-1.20.0/lib/sshkit/runners/parallel.rb:15:in `rescue in block (2 levels) in execute': Exception while executing as [email protected]: sidekiqctl exit status: 1 (SSHKit::Runner::ExecuteError)                                    
sidekiqctl stdout: Nothing written
sidekiqctl stderr: /home/deploy/.rvm/rubies/ruby-2.5.3/lib/ruby/site_ruby/2.5.0/bundler/rubygems_integration.rb:462:in `block in replace_bin_path': can't find executable sidekiqctl for gem sidekiq (Gem::Exception)                                                                      
        from /home/deploy/.rvm/rubies/ruby-2.5.3/lib/ruby/site_ruby/2.5.0/bundler/rubygems_integration.rb:482:in `block in replace_bin_path' 
        from /home/deploy/app_staging/shared/bundle/ruby/2.5.0/bin/sidekiqctl:23:in `<main>'                                                

after updating to sidekiq6 and deploying app

[tcd]

@seuros ping ping

[ghost]

Any news on this?

[ghost]

(this was just for fun btw, it's easier to complain than help)

[jules-w2]

@seuros ping

[rwojnarowski]

This gem does support systemd and sidekiq 6.0 but currently sidekiq is locked to version < 6, so I've created fork which allow to use newest sidekiq 6.0.3: https://github.com/rwojnarowski/capistrano-sidekiq

In short, to get this gem working with sidekiq 6.0: In Gemfile: gem "capistrano-sidekiq", git: "https://github.com/rwojnarowski/capistrano-sidekiq.git" In deploy.rb: set :init_system, :systemd On your server: loginctl enable-linger USERACCOUNT From repo directory run: bundle exec cap sidekiq:install You may need to enable new service on your server, so: systemctl --user enable sidekiq-production systemctl --user start sidekiq-production Check the status: systemctl --user status sidekiq-production and deploy new app as usual with Capistrano

[krtschmr]

@rwojnarowski well, my bundler path doesn't seem to be there tho

[ziaulrehman40]

@rwojnarowski Worked like a charm, hopefully main gem will be updated soon.

UPDATE:

status is giving me this:

● sidekiq-production.service - sidekiq for simplypo (production)
   Loaded: loaded (/home/user/.config/systemd/user/sidekiq-production.service; enabled; vendor preset: enabled)
   Active: failed (Result: exit-code) since Fri 2019-11-29 10:55:32 UTC; 15min ago
 Main PID: 8450 (code=exited, status=203/EXEC)

Nov 29 10:55:32 ip-172-30-0-142 systemd[1430]: sidekiq-production.service: Service hold-off time over, scheduling restart.
Nov 29 10:55:32 ip-172-30-0-142 systemd[1430]: sidekiq-production.service: Scheduled restart job, restart counter is at 5.
Nov 29 10:55:32 ip-172-30-0-142 systemd[1430]: Stopped sidekiq for MYAPP (production).
Nov 29 10:55:32 ip-172-30-0-142 systemd[1430]: sidekiq-production.service: Start request repeated too quickly.
Nov 29 10:55:32 ip-172-30-0-142 systemd[1430]: sidekiq-production.service: Failed with result 'exit-code'.
Nov 29 10:55:32 ip-172-30-0-142 systemd[1430]: Failed to start sidekiq for MYAPP (production).

Update 2:

I tracked it down to the path of bundler in generated service was not relative to rvm. I change that line to ExecStart=/home/deployer/.rvm/gems/ruby-2.6.5/wrappers/bundle exec sidekiq -e production -C config/sidekiq.yml and boom.

For anyone playing around with systemctl for sidekiq or anything, i used following sequence of commands for hit and try:

sudo nano /home/USER/.config/systemd/user/sidekiq-production.service
# make your changes

# do everything to restart :dagger:  
sudo systemctl daemon-reload && systemctl --user reenable sidekiq-production.service && systemctl --user stop sidekiq-production.service && systemctl --user start sidekiq-production.service && sleep 3 && systemctl --user status sidekiq-production.service

Commenting this to help others as well as my log for next time.

[mihir-kumar-thakur]

https://curationexperts.github.io/playbook/production/sidekiq_in_production.html

1. gem 'capistrano-sidekiq' , group: :development

2. Load the following in your Capfile require 'capistrano/sidekiq'

3. Install sidekiq as a systemd service on your server(s) Start by adding Sidekiq specific settings to your config/deploy.rb file

• Sidekiq service defaults set :init_system, :systemd set :service_unit_name, "sidekiq.service"

4. sudo loginctl enable-linger deploy

5. bundle exec cap $STAGE_NAME sidekiq:install after this you may go to server and check rbenv path /home/deploy/.config/systemd/user/sidekiq.service and check rbenv path

[agatheblues]

Newbies corner for those out here who are struggling with this 🤼‍♀️ I got it to work by re-reading that line and giving it the proper attention:

after this you may go to server and check rbenv path /home/deploy/.config/systemd/user/sidekiq.service and check rbenv path

This means: open that file and check that the ExecStart is using the correct path to your ruby bundle. If you're using rbenv, chances are, it is not the correct path.

ExecStart=/bin/bash -lc 'exec /home/deploy/.rbenv/shims/bundle exec sidekiq -e production'

Then, capistrano will tell you (check the logs, I'm learning that it's a healthy habit 🧘‍♀️ ) when deploying that this file got changed and you have to systemctl --user daemon-reload. Do that. Then ps aux | grep '[s]idekiq' should tell you that capistrano managed to start sidekiq like a breeze 🎉

[kannancet]

I struggled with this a lot. Downgraded to 5.2.7 without wasting time on systemctl and things worked!

[avvari-da]

This & This (together) worked for me! 🎉

[krthi]

Any update on the support for Sidekiq 6?

[ramonrails]

This & This (together) worked for me! 🎉

This still fails for me :( rbenv, ruby 2.7.1, sidekiq 6.1.2, capistrano-sidekiq master@/rwojnarowski, ExecStart updated to match rbenv bundle

sidekiq-production.service - sidekiq for commerce (production)
     Loaded: loaded (/home/deploy/.config/systemd/user/sidekiq-production.service; enabled; vendor preset: enabled)
     Active: failed (Result: exit-code) since Mon 2020-09-21 13:50:08 IST; 1s ago
    Process: 582875 ExecStart=/usr/local/rbenv/bin/rbenv exec bundle exec sidekiq -e production (code=exited, status=1/FAILURE)
   Main PID: 582875 (code=exited, status=1/FAILURE)

Sep 21 13:50:08 e2e-70-40 systemd[1829]: sidekiq-production.service: Scheduled restart job, restart counter is at 5.
Sep 21 13:50:08 e2e-70-40 systemd[1829]: Stopped sidekiq for commerce (production).
Sep 21 13:50:08 e2e-70-40 systemd[1829]: sidekiq-production.service: Start request repeated too quickly.
Sep 21 13:50:08 e2e-70-40 systemd[1829]: sidekiq-production.service: Failed with result 'exit-code'.
Sep 21 13:50:08 e2e-70-40 systemd[1829]: Failed to start sidekiq for commerce (production).

working directory is also set

WorkingDirectory=/var/www/<project>/commerce/current
ExecStart=/usr/local/rbenv/bin/rbenv exec bundle exec sidekiq -e production

but works when I run it from current_path manually at the server

deploy@e2e-70-40:/var/www/<project>/commerce/current$ /usr/local/rbenv/bin/rbenv exec bundle exec sidekiq -e production
2020-09-21T08:31:14.057Z pid=583216 tid=ck34 INFO: Booting Sidekiq 6.1.2 with redis options {:url=>"redis://localhost:6379/0"}
2020-09-21T08:31:14.403Z pid=583216 tid=ck34 INFO: Booted Rails 6.0.3.3 application in production environment
2020-09-21T08:31:14.403Z pid=583216 tid=ck34 INFO: Running in ruby 2.7.1p83 (2020-03-31 revision a0c7c23c9c) [x86_64-linux]
2020-09-21T08:31:14.403Z pid=583216 tid=ck34 INFO: See LICENSE and the LGPL-3.0 for licensing details.
2020-09-21T08:31:14.404Z pid=583216 tid=ck34 INFO: Upgrade to Sidekiq Pro for more features and support: https://sidekiq.org

[gauravm31]

@leoarnold I followed your steps. Created a sidekiq.service file and tried to use that in deploy.rb.

I cant seem to get this working. While deploying, capistrano cant find the service file.

On my server, i can start sidekiq manually with systemctl start sidekiq. It asks for user authentication but then works. If I do systemctl --user start sidekiq, it says "Failed to restart sidekiq.service: Unit sidekiq.service not found".

Any idea about this.

[leoarnold]

@gauravm31 That is because you installed the systemd unit in system space, not in user space. To run services in user space, enable-linger for the unprivileged user and then follow e.g. https://computingforgeeks.com/how-to-run-systemd-service-without-root-sudo/

[gauravm31]

@leoarnold Followed the steps. Then there was another issue regarding user/group. If we user systemctl with --user, we need to remove the user and group setting from service file as it is not needed. Otherwise it breaks. explained here https://unix.stackexchange.com/questions/438064/

after removing user and group settings, it works fine. Thanks for the help.