LoadError - aws_backend when trying to run Custom Resource

[michellehodges]

Trying to create a custom resource for Cloudfront, presented with a load error for aws_backend.

Describe the problem

When trying to run inspec on new custom resource (cloudfront), I'm presented with this error: /opt/inspec/embedded/lib/ruby/2.6.0/rubygems/core_ext/kernel_require.rb:54:in require': cannot load such file -- aws_backend (LoadError)

I have confirmed that inspec works fine with pre-built functionalities such as S3 bucket, and that the inspec package exist locally, including the file aws_backend.rb.

Solutions that I've tried:

• Upgraded inspec version to 4.19.0
• Did various changes to inspec.yml file (I did delete inspec.lock every time I did a change), with this being the current:

inspec_version: '>= 4.18.114'
depends:
  - name: inspec-aws
    url: https://github.com/inspec/inspec-aws/archive/v1.19.0.zip
supports:
  - platform: aws

Full Error

Traceback (most recent call last):
	23: from /usr/local/bin/inspec:210:in <main>'
	22: from /usr/local/bin/inspec:210:in load'
	21: from /opt/inspec/embedded/lib/ruby/gems/2.6.0/gems/inspec-bin-4.19.0/bin/inspec:11:in <top (required)>'
	20: from /opt/inspec/embedded/lib/ruby/gems/2.6.0/gems/inspec-core-4.19.0/lib/inspec/base_cli.rb:35:in start'
	19: from /opt/inspec/embedded/lib/ruby/gems/2.6.0/gems/thor-0.20.3/lib/thor/base.rb:466:in start'
	18: from /opt/inspec/embedded/lib/ruby/gems/2.6.0/gems/thor-0.20.3/lib/thor.rb:387:in dispatch'
	17: from /opt/inspec/embedded/li/ruby/gems/2.6.0/gems/thor-0.20.3/lib/thor/invocation.rb:126:in invoke_command'
	16: from /opt/inspec/embedded/lib/ruby/gems/2.6.0/gems/thor-0.20.3/lib/thor/command.rb:27:in run'
	15: from /opt/inspec/embedded/lib/ruby/gems/2.6.0/gems/inspec-core-4.19.0/lib/inspec/cli.rb:279:in exec'
	14: from /opt/inspec/embedded/lib/ruby/gems/2.6.0/gems/inspec-core-4.19.0/lib/inspec/runner.rb:129:in run'
	13: from /opt/inspec/embedded/lib/ruby/gems/2.6.0/gems/inspec-core-4.19.0/lib/inspec/runner.rb:101:in load'
	12: from /opt/inspec/embedded/lib/ruby/gems/2.6.0/gems/inspec-core-4.19.0/lib/inspec/runner.rb:101:in each'
	11: from /opt/inspec/embedded/lib/ruby/gems/2.6.0/gems/inspec-core-4.19.0/lib/inspec/runner.rb:107:in block in load'
	10: from /opt/inspec/embedded/lib/ruby/gems/2.6.0/gems/inspec-core-4.19.0/lib/inspec/profile.rb:273:in load_libraries'
	 9: from /opt/inspec/embedded/lib/ruby/gems/2.6.0/gems/inspec-core-4.19.0/lib/inspec/profile_context.rb:143:in load_libraries'
	 8: from /opt/inspec/embedded/lib/ruby/gems/2.6.0/gems/inspec-core-4.19.0/lib/inspec/profile_context.rb:143:in each'
	 7: from /opt/inspec/embedded/lib/ruby/gems/2.6.0/gems/inspec-core-4.19.0/lib/inspec/profile_context.rb:144:in block in load_libraries'
	 6: from /opt/inspec/embedded/lib/ruby/gems/2.6.0/gems/inspec-core-4.19.0/lib/inspec/profile_context.rb:159:in load_library_file'
	 5: from /opt/inspec/embedded/lib/ruby/gems/2.6.0/gems/inspec-core-4.19.0/lib/inspec/profile_context.rb:170:in load_with_context'
	 4: from /opt/inspec/embedded/lib/ruby/gems/2.6.0/gems/inspec-core-4.19.0/lib/inspec/profile_context.rb:170:in instance_eval'
	 3: from libraries/aws_cloudfront.rb:3:in load_with_context'
	 2: from /opt/inspec/embedded/lib/ruby/gems/2.6.0/gems/inspec-core-4.19.0/lib/inspec/dsl_shared.rb:33:in require'
	 1: from /opt/inspec/embedded/lib/ruby/2.6.0/rubygems/core_ext/kernel_require.rb:54:in require'
/opt/inspec/embedded/lib/ruby/2.6.0/rubygems/core_ext/kernel_require.rb:54:in require': cannot load such file -- aws_backend (LoadError)

[omerdemirok]

@michellehodges If you are creating a resource locally, your inspec.yml should point your local directory where you created your new resource. See here for an example: https://github.com/inspec/inspec-aws/blob/master/test/integration/verify/inspec.yml

[michellehodges]

That doesnt work either, because now I'm presented with this issue: cannot load such file -- aws-sdk-cloudfront

Seems like the problem that I have here is that the aws_backend.rb file in the inspec-aws library is missing aws-sdk-cloudfront in its requirements and gem file.

Since we cannot pull the aws-sdk-cloudfront gem into our custom resource we would need to either:

1. Make changes to the aws_backend.rb file and the gemfile to bring in aws-sdk-cloudfront, then submit a merge request to inspec,
2. OR create a plugin that pulls in the proper gem, aws-sdk-cloudfront, then create a process to load this plugin at runtime + use a copy of aws_backend.rb that imports this gem to create the cloudfront_client functionality (which means that this will be maintained manually, instead of using latest versions of chef inspec)

Either way, neither of these solutions imply an easy way to implement custom resources... Please advise.

[omerdemirok]

@michellehodges, aws-sdk-cloudfront will be available in InSpec through train-aws plugin when this https://github.com/inspec/train-aws/pull/41 change is merged and released with InSpec thanks to your effort.

You can still continue developing the resource locally by adding gem "aws-sdk-cloudfront", "~> 1.0" in this file.

As you mentioned above, you should update the aws_backend.rb to require the new SDK and create a new client in AwsConnection class to be used in your resource.

When you are ready for a pull request, please remove the gem "aws-sdk-cloudfront", "~> 1.0" from the Gemfile in inspec-aws. Your new resource will be merged into the resource pack when the necessary SDK is available through new version of InSpec.

I hope this unblocks the issue for now.

[aaronlippold]

This is also causing issue with local custom resource in profiles. Guidance on how to correctly load the aws_backend from the resource pack in profile local resources would be much appricated as well.