Null public_ip_address expectation fails exec, succeeds in shell
inspec exec and inspec shell have different behavior for the same code that checks the existence of the public IP address.
❯ inspec version
4.37.17
Profile: Amazon Web Services Resource Pack (inspec-aws)
Version: 1.42.0
macOS Catalina 10.15.7
Describe the problem
I deployed an EC2 instance named database in a private subnet, with no public IP address. The following example tests that fact:
describe aws_ec2_instance(name: 'database') do
its('public_ip_address') { should_not be }
end
When I run it via inspec shell, the result is as expected:
❯ inspec shell -t aws://us-east-1
Welcome to the interactive InSpec Shell
To find out how to use it, type: help
You are currently running on:
Name: aws
Families: cloud, api
Release: train-aws: v0.2.0, aws-sdk-core: v3.114.0
inspec> describe aws_ec2_instance(name: 'database') do
inspec> its('public_ip_address') { should_not be }
inspec> end
Profile: inspec-shell
Version: (not specified)
EC2 Instance database
✔ public_ip_address is expected not to be
Test Summary: 1 successful, 0 failures, 0 skipped
However, if I run the exact same code via inspec exec, the result is different and causes a test failure:
❯ inspec exec aws-security -t aws://us-east-1
Profile: AWS InSpec Profile (aws-security)
Version: 0.1.0
Target: aws://us-east-1
EC2 Instance database
× public_ip_address is expected not to be
expected #<#<Class:0x00007fe936edd1c8>::NullResponse:0x00007fe93f0f6880> to evaluate to false
Profile: Amazon Web Services Resource Pack (inspec-aws)
Version: 1.42.0
Target: aws://us-east-1
No tests executed.
Test Summary: 0 successful, 1 failure, 0 skipped
When I add other tests, they appear to be consistent between the two methods.
(I came across this problem during the introductory course at https://learn.chef.io/courses/course-v1:chef+Inspec101+Perpetual/about. The behavior appears while following the steps in the Verify the Infrastructure section. The behavior appears to be newer than the course materials, as the error was not shown in the output listed in the course.)