openssl issues when running from jenkins on systemd

[caarlos0]

Description

This is a weird case, so I'm not sure if here is the right place...

Anyway, I have a Jenkins instance which runs several jobs (yeah, I know that the start of this story is already bad), some of them call chef, berks and etc.

We already had a very weird issue a few months ago related to kitchen docker driver and the container naming, which we worked around without finding out what's happening. +info

If we start jenkins using its provided init script (service jenkins start -> systemd calls the sysvinit script underneath and start it up), we get errors like Unsupported digest algorithm (MD5) on berks vendor and also on kitchen test.

If I copy the java -jar cmd line and start it manually, everything works.

ChefDK Version

# chef --version
Chef Development Kit Version: 3.6.57
chef-client version: 14.8.12
delivery version: master (5fb4908da53579c9dcf4894d4acf94d2d9ee6475)
berks version: 7.0.7
kitchen version: 1.24.0
inspec version: 3.2.6

Platform Version

# cat /etc/lsb-release
DISTRIB_ID=Ubuntu
DISTRIB_RELEASE=16.04
DISTRIB_CODENAME=xenial
DISTRIB_DESCRIPTION="Ubuntu 16.04.5 LTS"

Replication Case

That's the problem, I don't have one.

Stacktrace

An error occurred while reading the Berksfile:

  Unsupported digest algorithm (MD5).
	/opt/chefdk/embedded/lib/ruby/2.5.0/openssl/digest.rb:40:in `initialize'
	/opt/chefdk/embedded/lib/ruby/2.5.0/openssl/digest.rb:40:in `block (3 levels) in <class:Digest>'
	/opt/chefdk/embedded/lib/ruby/gems/2.5.0/gems/chef-14.8.12/lib/chef/digester.rb:53:in `new'
	/opt/chefdk/embedded/lib/ruby/gems/2.5.0/gems/chef-14.8.12/lib/chef/digester.rb:53:in `generate_md5_checksum_for_file'
	/opt/chefdk/embedded/lib/ruby/gems/2.5.0/gems/chef-14.8.12/lib/chef/digester.rb:49:in `generate_md5_checksum_for_file'
	/opt/chefdk/embedded/lib/ruby/gems/2.5.0/gems/chef-14.8.12/lib/chef/cookbook_version.rb:80:in `checksum_cookbook_file'
	/opt/chefdk/embedded/lib/ruby/gems/2.5.0/gems/chef-14.8.12/lib/chef/cookbook_manifest.rb:307:in `checksum_cookbook_file'
	/opt/chefdk/embedded/lib/ruby/gems/2.5.0/gems/chef-14.8.12/lib/chef/cookbook_manifest.rb:238:in `block in generate_manifest'
	/opt/chefdk/embedded/lib/ruby/gems/2.5.0/gems/chef-14.8.12/lib/chef/cookbook_manifest.rb:233:in `each'
	/opt/chefdk/embedded/lib/ruby/gems/2.5.0/gems/chef-14.8.12/lib/chef/cookbook_manifest.rb:233:in `generate_manifest'
	/opt/chefdk/embedded/lib/ruby/gems/2.5.0/gems/chef-14.8.12/lib/chef/cookbook_manifest.rb:104:in `manifest'
	/opt/chefdk/embedded/lib/ruby/gems/2.5.0/gems/chef-14.8.12/lib/chef/cookbook_manifest.rb:175:in `files_for'
	/opt/chefdk/embedded/lib/ruby/gems/2.5.0/gems/chef-14.8.12/lib/chef/cookbook_version.rb:177:in `fully_qualified_recipe_names'
	/opt/chefdk/embedded/lib/ruby/gems/2.5.0/gems/chef-14.8.12/lib/chef/cookbook/metadata.rb:374:in `recipes_from_cookbook_version'
	/opt/chefdk/embedded/lib/ruby/gems/2.5.0/gems/chef-14.8.12/lib/chef/cookbook_version.rb:154:in `metadata='
	/opt/chefdk/embedded/lib/ruby/gems/2.5.0/gems/chef-14.8.12/lib/chef/cookbook/cookbook_version_loader.rb:100:in `block in cookbook_version'
	/opt/chefdk/embedded/lib/ruby/gems/2.5.0/gems/chef-14.8.12/lib/chef/cookbook/cookbook_version_loader.rb:98:in `tap'
	/opt/chefdk/embedded/lib/ruby/gems/2.5.0/gems/chef-14.8.12/lib/chef/cookbook/cookbook_version_loader.rb:98:in `cookbook_version'
	/opt/chefdk/embedded/lib/ruby/gems/2.5.0/gems/berkshelf-7.0.7/lib/berkshelf/berksfile.rb:183:in `metadata'
	/opt/chefdk/embedded/lib/ruby/gems/2.5.0/gems/cleanroom-1.0.0/lib/cleanroom.rb:130:in `public_send'
	/opt/chefdk/embedded/lib/ruby/gems/2.5.0/gems/cleanroom-1.0.0/lib/cleanroom.rb:130:in `block (3 levels) in cleanroom'
	/var/lib/jenkins/workspace/KitchenTest_builds/cookbooks/labs_dependencies/Berksfile:28:in `evaluate'
	/opt/chefdk/embedded/lib/ruby/gems/2.5.0/gems/cleanroom-1.0.0/lib/cleanroom.rb:70:in `instance_eval'
	/opt/chefdk/embedded/lib/ruby/gems/2.5.0/gems/cleanroom-1.0.0/lib/cleanroom.rb:70:in `evaluate'
	/opt/chefdk/embedded/lib/ruby/gems/2.5.0/gems/cleanroom-1.0.0/lib/cleanroom.rb:56:in `evaluate_file'
	/opt/chefdk/embedded/lib/ruby/gems/2.5.0/gems/cleanroom-1.0.0/lib/cleanroom.rb:173:in `evaluate_file'
	/opt/chefdk/embedded/lib/ruby/gems/2.5.0/gems/berkshelf-7.0.7/lib/berkshelf/berksfile.rb:27:in `from_file'
	/opt/chefdk/embedded/lib/ruby/gems/2.5.0/gems/berkshelf-7.0.7/lib/berkshelf/berksfile.rb:16:in `from_options'
	/opt/chefdk/embedded/lib/ruby/gems/2.5.0/gems/berkshelf-7.0.7/lib/berkshelf/cli.rb:375:in `vendor'
	/root/.chefdk/gem/ruby/2.5.0/gems/thor-0.20.3/lib/thor/command.rb:27:in `run'
	/root/.chefdk/gem/ruby/2.5.0/gems/thor-0.20.3/lib/thor/invocation.rb:126:in `invoke_command'
	/root/.chefdk/gem/ruby/2.5.0/gems/thor-0.20.3/lib/thor.rb:387:in `dispatch'
	/opt/chefdk/embedded/lib/ruby/gems/2.5.0/gems/berkshelf-7.0.7/lib/berkshelf/cli.rb:47:in `dispatch'
	/root/.chefdk/gem/ruby/2.5.0/gems/thor-0.20.3/lib/thor/base.rb:466:in `start'
	/opt/chefdk/embedded/lib/ruby/gems/2.5.0/gems/berkshelf-7.0.7/lib/berkshelf/cli.rb:23:in `execute!'
	/opt/chefdk/embedded/lib/ruby/gems/2.5.0/gems/berkshelf-7.0.7/bin/berks:5:in `<top (required)>'
	/opt/chefdk/bin/berks:306:in `load'
	/opt/chefdk/bin/berks:306:in `<main>'
Makefile:4: recipe for target 'vendor' failed

I also did check, running from jenkins:

+ openssl version
OpenSSL 1.0.2q  20 Nov 2018
[Pipeline] sh
+ ruby -ropenssl -e p OpenSSL::OPENSSL_VERSION
"OpenSSL 1.0.2q  20 Nov 2018"
[Pipeline] sh
+ ruby -ropenssl -e p OpenSSL::X509::DEFAULT_CERT_FILE
"/opt/chefdk/embedded/ssl/cert.pem"

Seems like the correct version of openssl and all.

Before this berks issue started, we were running chefdk 3.1.0. Back then, the only issue we had was the kitchen name thing. When we upgraded, we started to see the berks issue too.

I'm happy to provide any more info you think is relevant... I've looked at so many things I'm not sure what really matters and what doesn't. Let me know!

Thanks

[tas50]

Are you on a system with FIPS enabled?

[caarlos0]

Are you on a system with FIPS enabled?

AFAIK no, how can I double check?

[tyler-ball]

@caarlos0 Check for the existence of /proc/sys/crypto/fips_enabled and if it exists whether it contains a 1. Lack of existence or a 0 in that file mean the system is not running in FIPS mode.

We ask because the OpenSSL FIPS module does not support MD5. Otherwise it is weird to see that not being supported. Lets check that first and if that is not the issue we will keep troubleshooting.