chef
chef-client hardcoded aws-sdk-core causing gem activation conflict
Internal issue link CHEF-5464
Description
On May-31 there was change introduced in aws-sdk-cloudformation and other packages https://github.com/aws/aws-sdk-ruby/blob/version-3/gems/aws-sdk-cloudformation/aws-sdk-cloudformation.gemspec#L28
$ md5sum chef_18.2.7-1_amd64.deb 5915e85e520fbe35b4d38f04e91bbc79 chef_18.2.7-1_amd64.deb
there opt/chef/bin/chef-client file contains line
gem "aws-sdk-core", "= 3.171.0"
If we tried in our system to use aws-sdk-core 3.174.0 then we got message mentioned in Output section
If we tried to fix on 3.171.0, then also packages like aws-sdk-lambda, aws-sdk-ec2, aws-sdk-autoscaling and aws-sdk-cloudformation needed downgraded fixed versions.
Current workaround
After installing chef and aws-sdk-core and before chef executions we changed hardcoded version in bin/chef-client:
sudo sed -i "s/3.171.0/3.174.0/g" /opt/chef/bin/chef-client
after that we were able to execute cloudformation ami build.
Suggestion
Update chef-client to use aws-sdk-core version 3.174.0 (or maybe even better if it would not have hardcoded version at all)
Chef Version
Initially 18.1.0 but repeatable also with 18.2.7
Platform Version
Ubuntu 20.04.6 LTS
Replication Case
Client Output
[ 65.255520] cloud-init[1224]: [2023-06-14T12:02:57+00:00] FATAL: Gem::ConflictError: Unable to activate aws-sdk-cloudformation-1.80.0, because aws-sdk-core-3.171.0 conflicts with aws-sdk-core (~> 3, >= 3.174.0)
Stacktrace
: Installing cookbook gem dependencies:
: Compiling cookbooks...
: Loading Chef InSpec profile files:
: Loading Chef InSpec input files:
: Loading Chef InSpec waiver files:
: --- ERROR[#get_meta_data]: Request to metadata endpoint public-ipv4 failed. The function will return empty string.
: [2023-06-14T12:02:56+00:00] WARN: Resource apt_preference built into Chef Infra Client is being overridden by the resource from a cookbook. Please upgrade your cookbook or remove the cookbook from your run_list.
: Recipe: provision::monitoring-worker
: * chef_gem[aws-sdk-cloudformation] action install (up to date)
: * chef_gem[aws-sdk-kms] action install (up to date)
:
: ================================================================================
: Recipe Compile Error in /etc/chef/local-mode-cache/cache/cookbooks/provision/recipes/monitoring-worker.rb
: ================================================================================
:
: Gem::ConflictError
: ------------------
: Unable to activate aws-sdk-cloudformation-1.80.0, because aws-sdk-core-3.171.0 conflicts with aws-sdk-core (~> 3, >= 3.174.0)
:
: Cookbook Trace: (most recent call first)
: ----------------------------------------
: /etc/chef/local-mode-cache/cache/cookbooks/provision/recipes/monitoring-worker.rb:11:in `from_file'
:
: Relevant File Content:
: ----------------------
: /etc/chef/local-mode-cache/cache/cookbooks/provision/recipes/monitoring-worker.rb:
:
: 4: chef_gem "aws-sdk-cloudformation" do
: 5: compile_time true
: 6: end
: 7: chef_gem "aws-sdk-kms" do
: 8: compile_time true
: 9: end
: 10: require "aws-sdk-kms"
: 11>> require "aws-sdk-cloudformation"
: 12:
: 13: #======= SET CONSTANTS ====================================
: 14: REGION = EC2MetaData.get("placement/region")
: 15: LOGICAL_RECOURCE_ID = "EC2Instance".freeze
: 16: STACK_NAME = ENV["STACK_NAME"]
: 17: UNIQUE_ID = EC2MetaData.get("instance-id")
: 18: PARAMETER_PREFIX = ENV["PARAMETER_PREFIX"]
: 19:
: 20: cfn_message = {
:
: System Info:
: ------------
: chef_version=18.2.7
: platform=ubuntu
: platform_version=20.04
: ruby=ruby 3.1.2p20 (2022-04-12 revision 4491bb740a) [x86_64-linux]
: program_name=/usr/bin/chef-client
: executable=/opt/chef/bin/chef-client
:
:
: Running handlers:
: [2023-06-14T12:02:57+00:00] ERROR: Running exception handlers
: Running handlers complete
: [2023-06-14T12:02:57+00:00] ERROR: Exception handlers complete
: Infra Phase failed. 0 resources updated in 08 seconds
: [2023-06-14T12:02:57+00:00] FATAL: Stacktrace dumped to /etc/chef/local-mode-cache/cache/chef-stacktrace.out
: [2023-06-14T12:02:57+00:00] FATAL: ---------------------------------------------------------------------------------------
: [2023-06-14T12:02:57+00:00] FATAL: PLEASE PROVIDE THE CONTENTS OF THE stacktrace.out FILE (above) IF YOU FILE A BUG REPORT
: [2023-06-14T12:02:57+00:00] FATAL: ---------------------------------------------------------------------------------------
: [2023-06-14T12:02:57+00:00] FATAL: Gem::ConflictError: Unable to activate aws-sdk-cloudformation-1.80.0, because aws-sdk-core-3.171.0 conflicts with aws-sdk-core (~> 3, >= 3.174.0)
I'm also impacted by this. The aws gemfiles specifying a version conflicts with the chef gem version making it a bit of a hassle to track down the correct version of each aws sdk gem so it doesn't cause a conflict with chef's aws-sdk-core bundled version.
18.4.2 and still hitting this problem :/
[2024-04-09T19:42:44+00:00] FATAL: Gem::ConflictError: Unable to activate aws-sdk-ssm-1.166.0, because aws-sdk-core-3.171.0 conflicts with aws-sdk-core (~> 3, >= 3.191.0)
Other than trying to figure out (and hardcode) the service-specific gem version that matches the embedded aws-sdk-core gem, Is there another approach that makes more sense to use the AWS sdk with (for?) a gem isn't already bundled with chef-client?
@rjhornsby my apologies, I thought I at least merged these tests in after the rest of the conflict was resolved but I've opened a new PR to specifically validate that whatever gem strategy we're using doesn't break AWS gems https://github.com/chef/chef/pull/14358 ... currently they appear to be failing in kitchen-tests across the board