puppet-check icon indicating copy to clipboard operation
puppet-check copied to clipboard

Published 20 hours ago • verified publisher icon mschuchard

Errors running puppet-check when metadata.json misses requirements

Open tobru opened this issue 8 years ago • 6 comments

In one of our Puppet modules puppet-check fails with the following errors:

bash-4.3# puppet-check .
/usr/local/bundle/gems/puppet-check-1.3.2/lib/puppet-check/data_parser.rb:107:in `block (2 levels) in json': undefined method `empty?' for nil:NilClass (NoMethodError)
	from /usr/local/bundle/gems/puppet-check-1.3.2/lib/puppet-check/data_parser.rb:105:in `each'
	from /usr/local/bundle/gems/puppet-check-1.3.2/lib/puppet-check/data_parser.rb:105:in `block in json'
	from /usr/local/bundle/gems/puppet-check-1.3.2/lib/puppet-check/data_parser.rb:34:in `each'
	from /usr/local/bundle/gems/puppet-check-1.3.2/lib/puppet-check/data_parser.rb:34:in `json'
	from /usr/local/bundle/gems/puppet-check-1.3.2/lib/puppet-check.rb:78:in `execute_parsers'
	from /usr/local/bundle/gems/puppet-check-1.3.2/lib/puppet-check.rb:36:in `run'
	from /usr/local/bundle/gems/puppet-check-1.3.2/lib/puppet-check/cli.rb:13:in `run'
	from /usr/local/bundle/gems/puppet-check-1.3.2/bin/puppet-check:5:in `<top (required)>'
	from /usr/local/bundle/bin/puppet-check:22:in `load'
	from /usr/local/bundle/bin/puppet-check:22:in `<main>'

or

bash-4.3# puppet-check --style .
wrong number of arguments (5 for 1..3)
/usr/local/bundle/gems/puppet-3.8.7/lib/puppet/vendor/safe_yaml/lib/safe_yaml.rb:162:in `safe_load'
/usr/local/bundle/gems/rubocop-0.47.1/lib/rubocop/config_loader.rb:171:in `yaml_safe_load'
/usr/local/bundle/gems/rubocop-0.47.1/lib/rubocop/config_loader.rb:154:in `load_yaml_configuration'
/usr/local/bundle/gems/rubocop-0.47.1/lib/rubocop/config_loader.rb:34:in `load_file'
/usr/local/bundle/gems/rubocop-0.47.1/lib/rubocop/config_loader.rb:107:in `configuration_from_file'
/usr/local/bundle/gems/rubocop-0.47.1/lib/rubocop/config_store.rb:44:in `for'
/usr/local/bundle/gems/rubocop-0.47.1/lib/rubocop/runner.rb:163:in `cached_run?'
/usr/local/bundle/gems/rubocop-0.47.1/lib/rubocop/runner.rb:59:in `ensure in inspect_files'
/usr/local/bundle/gems/rubocop-0.47.1/lib/rubocop/runner.rb:61:in `inspect_files'
/usr/local/bundle/gems/rubocop-0.47.1/lib/rubocop/runner.rb:36:in `run'
/usr/local/bundle/gems/rubocop-0.47.1/lib/rubocop/cli.rb:72:in `execute_runner'
/usr/local/bundle/gems/rubocop-0.47.1/lib/rubocop/cli.rb:27:in `run'
/usr/local/bundle/gems/puppet-check-1.3.2/lib/puppet-check/ruby_parser.rb:21:in `block (2 levels) in ruby'
/usr/local/bundle/gems/puppet-check-1.3.2/lib/puppet-check/utils.rb:7:in `capture_stdout'
/usr/local/bundle/gems/puppet-check-1.3.2/lib/puppet-check/ruby_parser.rb:21:in `block in ruby'
/usr/local/bundle/gems/puppet-check-1.3.2/lib/puppet-check/ruby_parser.rb:8:in `each'
/usr/local/bundle/gems/puppet-check-1.3.2/lib/puppet-check/ruby_parser.rb:8:in `ruby'
/usr/local/bundle/gems/puppet-check-1.3.2/lib/puppet-check.rb:72:in `execute_parsers'
/usr/local/bundle/gems/puppet-check-1.3.2/lib/puppet-check.rb:36:in `run'
/usr/local/bundle/gems/puppet-check-1.3.2/lib/puppet-check/cli.rb:13:in `run'
/usr/local/bundle/gems/puppet-check-1.3.2/bin/puppet-check:5:in `<top (required)>'
/usr/local/bundle/bin/puppet-check:22:in `load'
/usr/local/bundle/bin/puppet-check:22:in `<main>'
wrong number of arguments (5 for 1..3)
/usr/local/bundle/gems/puppet-3.8.7/lib/puppet/vendor/safe_yaml/lib/safe_yaml.rb:162:in `safe_load'
/usr/local/bundle/gems/rubocop-0.47.1/lib/rubocop/config_loader.rb:171:in `yaml_safe_load'
/usr/local/bundle/gems/rubocop-0.47.1/lib/rubocop/config_loader.rb:154:in `load_yaml_configuration'
/usr/local/bundle/gems/rubocop-0.47.1/lib/rubocop/config_loader.rb:34:in `load_file'
/usr/local/bundle/gems/rubocop-0.47.1/lib/rubocop/config_loader.rb:107:in `configuration_from_file'
/usr/local/bundle/gems/rubocop-0.47.1/lib/rubocop/config_store.rb:44:in `for'
/usr/local/bundle/gems/rubocop-0.47.1/lib/rubocop/runner.rb:163:in `cached_run?'
/usr/local/bundle/gems/rubocop-0.47.1/lib/rubocop/runner.rb:59:in `ensure in inspect_files'
/usr/local/bundle/gems/rubocop-0.47.1/lib/rubocop/runner.rb:61:in `inspect_files'
/usr/local/bundle/gems/rubocop-0.47.1/lib/rubocop/runner.rb:36:in `run'
/usr/local/bundle/gems/rubocop-0.47.1/lib/rubocop/cli.rb:72:in `execute_runner'
/usr/local/bundle/gems/rubocop-0.47.1/lib/rubocop/cli.rb:27:in `run'
/usr/local/bundle/gems/puppet-check-1.3.2/lib/puppet-check/ruby_parser.rb:21:in `block (2 levels) in ruby'
/usr/local/bundle/gems/puppet-check-1.3.2/lib/puppet-check/utils.rb:7:in `capture_stdout'
/usr/local/bundle/gems/puppet-check-1.3.2/lib/puppet-check/ruby_parser.rb:21:in `block in ruby'
/usr/local/bundle/gems/puppet-check-1.3.2/lib/puppet-check/ruby_parser.rb:8:in `each'
/usr/local/bundle/gems/puppet-check-1.3.2/lib/puppet-check/ruby_parser.rb:8:in `ruby'
/usr/local/bundle/gems/puppet-check-1.3.2/lib/puppet-check.rb:72:in `execute_parsers'
/usr/local/bundle/gems/puppet-check-1.3.2/lib/puppet-check.rb:36:in `run'
/usr/local/bundle/gems/puppet-check-1.3.2/lib/puppet-check/cli.rb:13:in `run'
/usr/local/bundle/gems/puppet-check-1.3.2/bin/puppet-check:5:in `<top (required)>'
/usr/local/bundle/bin/puppet-check:22:in `load'
/usr/local/bundle/bin/puppet-check:22:in `<main>'
wrong number of arguments (5 for 1..3)
/usr/local/bundle/gems/puppet-3.8.7/lib/puppet/vendor/safe_yaml/lib/safe_yaml.rb:162:in `safe_load'
/usr/local/bundle/gems/rubocop-0.47.1/lib/rubocop/config_loader.rb:171:in `yaml_safe_load'
/usr/local/bundle/gems/rubocop-0.47.1/lib/rubocop/config_loader.rb:154:in `load_yaml_configuration'
/usr/local/bundle/gems/rubocop-0.47.1/lib/rubocop/config_loader.rb:34:in `load_file'
/usr/local/bundle/gems/rubocop-0.47.1/lib/rubocop/config_loader.rb:107:in `configuration_from_file'
/usr/local/bundle/gems/rubocop-0.47.1/lib/rubocop/config_store.rb:44:in `for'
/usr/local/bundle/gems/rubocop-0.47.1/lib/rubocop/runner.rb:163:in `cached_run?'
/usr/local/bundle/gems/rubocop-0.47.1/lib/rubocop/runner.rb:59:in `ensure in inspect_files'
/usr/local/bundle/gems/rubocop-0.47.1/lib/rubocop/runner.rb:61:in `inspect_files'
/usr/local/bundle/gems/rubocop-0.47.1/lib/rubocop/runner.rb:36:in `run'
/usr/local/bundle/gems/rubocop-0.47.1/lib/rubocop/cli.rb:72:in `execute_runner'
/usr/local/bundle/gems/rubocop-0.47.1/lib/rubocop/cli.rb:27:in `run'
/usr/local/bundle/gems/puppet-check-1.3.2/lib/puppet-check/ruby_parser.rb:21:in `block (2 levels) in ruby'
/usr/local/bundle/gems/puppet-check-1.3.2/lib/puppet-check/utils.rb:7:in `capture_stdout'
/usr/local/bundle/gems/puppet-check-1.3.2/lib/puppet-check/ruby_parser.rb:21:in `block in ruby'
/usr/local/bundle/gems/puppet-check-1.3.2/lib/puppet-check/ruby_parser.rb:8:in `each'
/usr/local/bundle/gems/puppet-check-1.3.2/lib/puppet-check/ruby_parser.rb:8:in `ruby'
/usr/local/bundle/gems/puppet-check-1.3.2/lib/puppet-check.rb:72:in `execute_parsers'
/usr/local/bundle/gems/puppet-check-1.3.2/lib/puppet-check.rb:36:in `run'
/usr/local/bundle/gems/puppet-check-1.3.2/lib/puppet-check/cli.rb:13:in `run'
/usr/local/bundle/gems/puppet-check-1.3.2/bin/puppet-check:5:in `<top (required)>'
/usr/local/bundle/bin/puppet-check:22:in `load'
/usr/local/bundle/bin/puppet-check:22:in `<main>'
wrong number of arguments (5 for 1..3)
/usr/local/bundle/gems/puppet-3.8.7/lib/puppet/vendor/safe_yaml/lib/safe_yaml.rb:162:in `safe_load'
/usr/local/bundle/gems/rubocop-0.47.1/lib/rubocop/config_loader.rb:171:in `yaml_safe_load'
/usr/local/bundle/gems/rubocop-0.47.1/lib/rubocop/config_loader.rb:154:in `load_yaml_configuration'
/usr/local/bundle/gems/rubocop-0.47.1/lib/rubocop/config_loader.rb:34:in `load_file'
/usr/local/bundle/gems/rubocop-0.47.1/lib/rubocop/config_loader.rb:107:in `configuration_from_file'
/usr/local/bundle/gems/rubocop-0.47.1/lib/rubocop/config_store.rb:44:in `for'
/usr/local/bundle/gems/rubocop-0.47.1/lib/rubocop/runner.rb:163:in `cached_run?'
/usr/local/bundle/gems/rubocop-0.47.1/lib/rubocop/runner.rb:59:in `ensure in inspect_files'
/usr/local/bundle/gems/rubocop-0.47.1/lib/rubocop/runner.rb:61:in `inspect_files'
/usr/local/bundle/gems/rubocop-0.47.1/lib/rubocop/runner.rb:36:in `run'
/usr/local/bundle/gems/rubocop-0.47.1/lib/rubocop/cli.rb:72:in `execute_runner'
/usr/local/bundle/gems/rubocop-0.47.1/lib/rubocop/cli.rb:27:in `run'
/usr/local/bundle/gems/puppet-check-1.3.2/lib/puppet-check/ruby_parser.rb:21:in `block (2 levels) in ruby'
/usr/local/bundle/gems/puppet-check-1.3.2/lib/puppet-check/utils.rb:7:in `capture_stdout'
/usr/local/bundle/gems/puppet-check-1.3.2/lib/puppet-check/ruby_parser.rb:21:in `block in ruby'
/usr/local/bundle/gems/puppet-check-1.3.2/lib/puppet-check/ruby_parser.rb:8:in `each'
/usr/local/bundle/gems/puppet-check-1.3.2/lib/puppet-check/ruby_parser.rb:8:in `ruby'
/usr/local/bundle/gems/puppet-check-1.3.2/lib/puppet-check.rb:72:in `execute_parsers'
/usr/local/bundle/gems/puppet-check-1.3.2/lib/puppet-check.rb:36:in `run'
/usr/local/bundle/gems/puppet-check-1.3.2/lib/puppet-check/cli.rb:13:in `run'
/usr/local/bundle/gems/puppet-check-1.3.2/bin/puppet-check:5:in `<top (required)>'
/usr/local/bundle/bin/puppet-check:22:in `load'
/usr/local/bundle/bin/puppet-check:22:in `<main>'
wrong number of arguments (5 for 1..3)
/usr/local/bundle/gems/puppet-3.8.7/lib/puppet/vendor/safe_yaml/lib/safe_yaml.rb:162:in `safe_load'
/usr/local/bundle/gems/rubocop-0.47.1/lib/rubocop/config_loader.rb:171:in `yaml_safe_load'
/usr/local/bundle/gems/rubocop-0.47.1/lib/rubocop/config_loader.rb:154:in `load_yaml_configuration'
/usr/local/bundle/gems/rubocop-0.47.1/lib/rubocop/config_loader.rb:34:in `load_file'
/usr/local/bundle/gems/rubocop-0.47.1/lib/rubocop/config_loader.rb:107:in `configuration_from_file'
/usr/local/bundle/gems/rubocop-0.47.1/lib/rubocop/config_store.rb:44:in `for'
/usr/local/bundle/gems/rubocop-0.47.1/lib/rubocop/runner.rb:163:in `cached_run?'
/usr/local/bundle/gems/rubocop-0.47.1/lib/rubocop/runner.rb:59:in `ensure in inspect_files'
/usr/local/bundle/gems/rubocop-0.47.1/lib/rubocop/runner.rb:61:in `inspect_files'
/usr/local/bundle/gems/rubocop-0.47.1/lib/rubocop/runner.rb:36:in `run'
/usr/local/bundle/gems/rubocop-0.47.1/lib/rubocop/cli.rb:72:in `execute_runner'
/usr/local/bundle/gems/rubocop-0.47.1/lib/rubocop/cli.rb:27:in `run'
/usr/local/bundle/gems/puppet-check-1.3.2/lib/puppet-check/ruby_parser.rb:21:in `block (2 levels) in ruby'
/usr/local/bundle/gems/puppet-check-1.3.2/lib/puppet-check/utils.rb:7:in `capture_stdout'
/usr/local/bundle/gems/puppet-check-1.3.2/lib/puppet-check/ruby_parser.rb:21:in `block in ruby'
/usr/local/bundle/gems/puppet-check-1.3.2/lib/puppet-check/ruby_parser.rb:8:in `each'
/usr/local/bundle/gems/puppet-check-1.3.2/lib/puppet-check/ruby_parser.rb:8:in `ruby'
/usr/local/bundle/gems/puppet-check-1.3.2/lib/puppet-check.rb:72:in `execute_parsers'
/usr/local/bundle/gems/puppet-check-1.3.2/lib/puppet-check.rb:36:in `run'
/usr/local/bundle/gems/puppet-check-1.3.2/lib/puppet-check/cli.rb:13:in `run'
/usr/local/bundle/gems/puppet-check-1.3.2/bin/puppet-check:5:in `<top (required)>'
/usr/local/bundle/bin/puppet-check:22:in `load'
/usr/local/bundle/bin/puppet-check:22:in `<main>'
wrong number of arguments (5 for 1..3)
/usr/local/bundle/gems/puppet-3.8.7/lib/puppet/vendor/safe_yaml/lib/safe_yaml.rb:162:in `safe_load'
/usr/local/bundle/gems/rubocop-0.47.1/lib/rubocop/config_loader.rb:171:in `yaml_safe_load'
/usr/local/bundle/gems/rubocop-0.47.1/lib/rubocop/config_loader.rb:154:in `load_yaml_configuration'
/usr/local/bundle/gems/rubocop-0.47.1/lib/rubocop/config_loader.rb:34:in `load_file'
/usr/local/bundle/gems/rubocop-0.47.1/lib/rubocop/config_loader.rb:107:in `configuration_from_file'
/usr/local/bundle/gems/rubocop-0.47.1/lib/rubocop/config_store.rb:44:in `for'
/usr/local/bundle/gems/rubocop-0.47.1/lib/rubocop/runner.rb:163:in `cached_run?'
/usr/local/bundle/gems/rubocop-0.47.1/lib/rubocop/runner.rb:59:in `ensure in inspect_files'
/usr/local/bundle/gems/rubocop-0.47.1/lib/rubocop/runner.rb:61:in `inspect_files'
/usr/local/bundle/gems/rubocop-0.47.1/lib/rubocop/runner.rb:36:in `run'
/usr/local/bundle/gems/rubocop-0.47.1/lib/rubocop/cli.rb:72:in `execute_runner'
/usr/local/bundle/gems/rubocop-0.47.1/lib/rubocop/cli.rb:27:in `run'
/usr/local/bundle/gems/puppet-check-1.3.2/lib/puppet-check/ruby_parser.rb:21:in `block (2 levels) in ruby'
/usr/local/bundle/gems/puppet-check-1.3.2/lib/puppet-check/utils.rb:7:in `capture_stdout'
/usr/local/bundle/gems/puppet-check-1.3.2/lib/puppet-check/ruby_parser.rb:21:in `block in ruby'
/usr/local/bundle/gems/puppet-check-1.3.2/lib/puppet-check/ruby_parser.rb:8:in `each'
/usr/local/bundle/gems/puppet-check-1.3.2/lib/puppet-check/ruby_parser.rb:8:in `ruby'
/usr/local/bundle/gems/puppet-check-1.3.2/lib/puppet-check.rb:72:in `execute_parsers'
/usr/local/bundle/gems/puppet-check-1.3.2/lib/puppet-check.rb:36:in `run'
/usr/local/bundle/gems/puppet-check-1.3.2/lib/puppet-check/cli.rb:13:in `run'
/usr/local/bundle/gems/puppet-check-1.3.2/bin/puppet-check:5:in `<top (required)>'
/usr/local/bundle/bin/puppet-check:22:in `load'
/usr/local/bundle/bin/puppet-check:22:in `<main>'
/usr/local/bundle/gems/puppet-check-1.3.2/lib/puppet-check/data_parser.rb:107:in `block (2 levels) in json': undefined method `empty?' for nil:NilClass (NoMethodError)
	from /usr/local/bundle/gems/puppet-check-1.3.2/lib/puppet-check/data_parser.rb:105:in `each'
	from /usr/local/bundle/gems/puppet-check-1.3.2/lib/puppet-check/data_parser.rb:105:in `block in json'
	from /usr/local/bundle/gems/puppet-check-1.3.2/lib/puppet-check/data_parser.rb:34:in `each'
	from /usr/local/bundle/gems/puppet-check-1.3.2/lib/puppet-check/data_parser.rb:34:in `json'
	from /usr/local/bundle/gems/puppet-check-1.3.2/lib/puppet-check.rb:78:in `execute_parsers'
	from /usr/local/bundle/gems/puppet-check-1.3.2/lib/puppet-check.rb:36:in `run'
	from /usr/local/bundle/gems/puppet-check-1.3.2/lib/puppet-check/cli.rb:13:in `run'
	from /usr/local/bundle/gems/puppet-check-1.3.2/bin/puppet-check:5:in `<top (required)>'
	from /usr/local/bundle/bin/puppet-check:22:in `load'
	from /usr/local/bundle/bin/puppet-check:22:in `<main>'

We're using the Docker image found under https://hub.docker.com/r/vshn/puppetci/. It brings puppet-check pre-installed. The source Dockerfile which builds this image is here: https://github.com/vshn/docker-puppetci/blob/master/Dockerfile3

Thanks for your help.

tobru avatar Mar 03 '17 13:03 tobru

After some more investigation and code reading: at least the error undefined method empty?' for nil:NilClass (NoMethodError)occurs when themetadata.jsonmisses therequirementsblock. At leastpuppet-check` should warn about that and don't fail with such an error I guess.

For the other error wrong number of arguments (5 for 1..3) I haven't found a cause for now.

tobru avatar Mar 03 '17 14:03 tobru

First issue:

I checked the source code and, strangely enough, my comment dictates the correct behavior but the code doesn't seem to follow it. It is a quick fix.

Second issue:

It looks like rubocop is using a yaml gem that comes with puppet instead of the one that comes with ruby. This could be an issue with rvm/rbenv gem loading.

mschuchard avatar Mar 03 '17 15:03 mschuchard

First issue:

This actually revealed that i have redundant code in the related method, and I don't know how I missed that when it is the first Reek warning for the file.

Second issue:

Looking at it again, there is a possibility that you have an invalid .rubcop.yml. Give that another look to verify. Otherwise, this is some weird environment error that may best be solved on Stack Overflow.

mschuchard avatar Mar 03 '17 19:03 mschuchard

First issue fixed in 26f38317f8ac65833ef508965682388bee5f418a. I recommend directly patching data_parser.rb as part of the Dockerfile for the image your are building as it will be a while before 1.4.0 since that will be the 'octocatalog-diff' release.

mschuchard avatar Mar 03 '17 19:03 mschuchard

Thanks a lot for looking into it and I'm very sorry to have mixed two unrelated issues in one ticket.

Looking at it again, there is a possibility that you have an invalid .rubcop.yml. Give that another look to verify. Otherwise, this is some weird environment error that may best be solved on Stack Overflow.

There is no .rubcop.yml file available in this module. Seems to be related to https://github.com/bbatsov/rubocop/issues/3539 but I'm not sure how to fix this. I also discovered that it only happens in the Docker image with Puppet 3. The other image with Puppet 4 doesn't have this issue. And interestingly enough it isn't a general issue, for some modules tested by puppet-check the issue doesn't show up, but I couldn't find the differences until now.

tobru avatar Mar 06 '17 12:03 tobru

Yeah, it did seem like an environment issue. The stack trace makes it seem like Rubocop is trying to load in a config file, so if there was some way to block that it might fix the issue.

mschuchard avatar Mar 06 '17 14:03 mschuchard