semgrep-action icon indicating copy to clipboard operation
semgrep-action copied to clipboard

Prevent one bad rule from crashing the action

Open emjin opened this issue 2 years ago • 4 comments

Generally, users still want scans to complete even if one rule is bad so that they aren't blocked. Need a solution that warns users sufficiently loudly but allows them to push code

emjin avatar Mar 16 '22 21:03 emjin

Important question to figure out: how will users and r2c be alerted about rules being bad?

For instance, today's issue, the github token secret scanning rule broke. It's critical for our trustworthiness that Semgrep doesn't ever silently allow a github token to be committed. A rule not scanning is nearly as much of a problem as a rule matching.

underyx avatar Mar 16 '22 21:03 underyx

Are rules generally version-controlled in the registry? Maybe some kind of backoff with decreasing versions could be a good fallback to (1) not break everything when one rule breaks and (2) continue doing at least as good as a job before an upgrade.

ziyadedher avatar Mar 16 '22 21:03 ziyadedher

^ way back when we talked about having a "minimum version" field in rules; semgrep would discard any rules whose minimum version was above the current version. I think ditto for maximum versions.

nbrahms avatar Mar 16 '22 22:03 nbrahms

Discussion in semgrep: https://github.com/returntocorp/semgrep/issues/4828

emjin avatar Mar 17 '22 19:03 emjin