chainloop
chainloop copied to clipboard
feat: attestation usability improvements
I have a couple of usability suggestions that might be worth implementing
-
If we are using the discovery mode, i.e do not provide
kind, and ends up falling back toartifactkind, we should explain that it didn't fit any other material kind. -
If we pass explicitly a
kindwe should probably show the error message too, so instead of
chainloop att add --value ~/Downloads/jira.cyclonedx.json --kind SBOM_CYCLONEDX_JSON
ERR adding material: crafting material: invalid cyclonedx sbom file: unexpected material type
it should show
oesn't validate with http://cyclonedx.org/schema/bom-1.6.schema.json#
[I#/components/0] [S#/properties/components/items/$ref] doesn't validate with '/definitions/component'
[I#/components/0/licenses] [S#/definitions/component/properties/licenses/$ref] doesn't validate with '/definitions/licenseChoice'
[I#/components/0/licenses] [S#/definitions/licenseChoice/oneOf] oneOf failed
[I#/components/0/licenses/0] [S#/definitions/licenseChoice/oneOf/0/items/required] missing properties:
- We should be able to provide a name so we do not need to use the autogenerated one. Tracked by: https://github.com/chainloop-dev/chainloop/issues/911
wdyt?
### Tasks
- [ ] explain discovery mode resolution when falling back
- [ ] output error when it's not part of autodiscovery
- [ ] https://github.com/chainloop-dev/chainloop/issues/911
- [ ] Document attestation modes https://github.com/chainloop-dev/chainloop/issues/901
- [ ] Look into accepting invalid artifacts
Regarding the last topic, here's the task for it with in depth explanation: https://github.com/chainloop-dev/chainloop/issues/911
Thanks @migmartri. I'm afraid malformed documents are going to be a frequent issue. Given the fact that Chainloop's role is attesting, it might be reasonable to allow skipping the validation and still store it as a SBOM_CYCLONEDX_JSON, probably with an annotation validated=false. This way, Chainloop is attesting that the document exists but it's not validated so it cannot be fully trusted.
Thanks @migmartri. I'm afraid malformed documents are going to be a frequent issue. Given the fact that Chainloop's role is attesting, it might be reasonable to allow skipping the validation and still store it as a
SBOM_CYCLONEDX_JSON, probably with an annotationvalidated=false. This way, Chainloop is attesting that the document exists but it's not validated so it cannot be fully trusted.
Completely agree here. If a validation skip feature is implemented I think it should be explicit (user has to declare that is the desired outcome) and then that must be reflected in the metadata. My two cents.
Thanks for the feedback. I agree with you both. What I am not sure about is if this feature should be either.
- a)
skip validationor - b)
continue if validation error
I am leaning towards b since it can inform us whether the document was valid. If we skip the validation, we lose that info.
In other words, I'd implement something like att add --soft-validation to add the evidence regardless of the validation output. The result will contain two extra annotations
valid: true|falsesoft-validation: true|false`
These two annotations would allow users for example to create a policy that enforces not to use this feature, meaning soft-validation can't be true.
Wdyt?