cloudformation-guard icon indicating copy to clipboard operation
cloudformation-guard copied to clipboard

Exit status returned as `0` when parse errors occur during rule testing

Open mrinaudo-aws opened this issue 3 years ago • 0 comments

Describe the bug When I use the test subcommand, if there is a parse error occurring on the rules file, or on the test file or on both, the resulting exit status being returned is 0, that I'd expect to have when no parse errors and no validation errors occur instead. The validate subcommand worked as expected for me.

To Reproduce

  • rules file containing a parse error (missing } at the end):
let volumes = Resources.*[ Type == 'AWS::EC2::Volume' ]
rule sample_volume when %volumes !empty {
    %volumes.Properties {
       Size <= 10
    }
  • test file in YAML format (no parse errors in this example):
---
- input:
    Resources:
      SampleVolume:
        Type: AWS::EC2::Volume
        Properties:
          Size: 100
  expectations:
    rules:
      sample_volume: FAIL
  • command invocation, with output:
cfn-guard test -t test-rules.yaml -r test-rules.guard
Parse Error on ruleset file Parser Error when parsing rules file Parsing Error Error parsing file test-rules.guard at line 6 at column 1, when handling , fragment 

Issuing echo $? in a Bash shell after this command, returns 0.

Expected behavior The behavior I'd expect is an exit status greater than 0 when parse errors occur during rules testing with the test subcommand.

Additional context Running Guard version 2.0.2.

mrinaudo-aws avatar Jun 15 '21 19:06 mrinaudo-aws