sensu-chef
sensu-chef copied to clipboard
rabbitmq-server Fails to Start After Updating Erlang Packages
Current Behavior
Following an apt-get dist-upgrade
, the packages 'erlang-mode' and 'esl-erlang' were upgraded above the highest version which supports the available version of 'rabbitmq-server', causing RabbitMQ to crash on startup. No logs were produced.
Possible Solution
Ensure appropriately versioned Erlang packages are installed along side their corresponding RabbitMQ server.
Steps to Reproduce (for bugs)
- Existing Ubuntu 16.04 server with the following packages installed:
- rabbitmq-server=3.6.15-1
- esl-erlang=1:20.3
- erlang-mode=1:20.3-1
-
apt-get dist-upgrade
which took versions to the following:- esl-erlang=1:21.0.5
- erlang-mode=1:21.0.5-1
- Attempt to start rabbitmq-server service. Failure.
- Downgrade 'esl-erlang' and 'erlang-mode' to versions listed in Step 1
- rabbitmq-server service starts appropriately.
Context
Merely attempting to patch the system with updated packages from Ubuntu.
Your Environment
- Version of this cookbook used: 5.3.0
- Version of Sensu used: 1.4.3-1
- Version of Chef used: 14.3.37
- Operating System and version (e.g. CentOS 7, Ubuntu 14.04): Ubuntu 16.04
Tagging @majormoses per thread in Sensu Community Slack #help channel from 29 August 2018 @ 21:27 UTC.
This link may not work in the future (as we have limited retention on free slack) but here is the discussion: https://sensucommunity.slack.com/archives/C68LV5M9U/p1535578029000100
From a question in the Slack channel: The rabbitmq cookbook pulled in:
rabbitmq (5.6.1)
dpkg_autostart (>= 0.0.0)
erlang (>= 0.0.0)
logrotate (>= 0.0.0)
yum-epel (>= 0.0.0)
yum-erlang_solutions (>= 0.0.0)
Sorry what I needed was what version of the rabbitmq cookbook got pulled in, based on what I see (the version of rabbitmq itself) you are on at least 5.x
but I'd like to confirm.
Sorry, for whatever reason, when I put that in, the code block wasn't formatted properly and the rabbitmq line didn't show up. Fixed it.
Can you confirm if you modified this attribute? https://github.com/rabbitmq/chef-cookbook/blob/v5.x/attributes/default.rb#L6 from what I can tell from the defaults this should not have happened via an apt upgrade unless that was changed. You can set node['esl']['version']
similar to this: https://github.com/rabbitmq/chef-cookbook/blob/v5.x/.kitchen.yml#L29
I did not modify that attribute, no. Other than enabling TLS and setting a hostname and credentials using the attributes in this cookbook, I did not modify the RabbitMQ setup. I felt that the least amount that I could do to it would be best as I don't understand RabbitMQ well enough, and Redis wasn't going to be secure enough for my deployment pattern.
I opened a PR on the upstream erlang cookbook which might help your scenerio: https://github.com/chef-cookbooks/erlang/pull/65
@SeanSith you could try pulling in erlang cookbook 7.0.0
I think that should work, if you have a server to test with you can remove your hold and see if this prevents the issue from manifesting? The fix I made only fix debian based distros as you need some non standard repos to pin versions with yum
.