sbom-tool icon indicating copy to clipboard operation
sbom-tool copied to clipboard

SBOM validation returns success if signature validation fails

Open pownkel opened this issue 1 year ago • 0 comments

ValidationResultGenerator sets the "Success" value of a ValidationResult object to either true or false depending on whether the number of validation failures ValidationResultGenerator.cs@L86) is equal to zero. However, the list of validation failures is typed to only contain file validation failures, i.e., failures that happened while parsing and validating the contents of the SBOM. If a signing validation occurs, we simply log an error message to the console and exit (SBOMParserBasedValidationWorkflow.cs@L91). Signing validation occurs before any other validation, so if it fails, no files are ever attempted to be validated - which also means there are no file validation errors.

The result is that, if signing validation fails, the tool logs an error, correctly sets the exit code to a nonzero value, and then returns a ValidationResult with Success: true. This can be confusing for users. I can think of two possible solutions:

  1. Change the type of ValidationErrors to include a broader error type which can include signing failures
  2. Add a "SignatureValidated" field to ValidationResult indicating whether a signature was validated and/or what the result was, and take this value into account when setting the value of "Success"

I'm partial to option 1, since it's simpler and would minimize changes to our interface, but it would likely be a more significant code change. Worth noting is that ValidationResult also logs the configuration that was used to run the tool, including the "ValidateSignature" flag, so another field stating whether signature validation ran would be somewhat redundant (although it would be clearer)

EDIT: It looks like at some point there was an effort to migrate to EntityError instead of File error (see this comment), maybe this would be worth picking up again

pownkel avatar Feb 14 '24 00:02 pownkel