PolicyEngine's ALL operator is flawed

[eldis80]

The policy evaluation with operator 'All' doesn't work as expected. The problem is that for the 'All' operator to work, each evaluator should return at most one violation. This is because in the code the number of violations is compared to the number of conditions.

Here: https://github.com/DependencyTrack/dependency-track/blob/7685a29143dbb23f6d63af20ca4845790d81023b/src/main/java/org/dependencytrack/policy/PolicyEngine.java#L92

At least I'm expecting the 'All' operator to mean that every conditions must match for a policy violation to be raised. We noticed this issue when trying to implement a policy which would include only critical application vulnerabilities and exclude critical OS (Debian/Ubuntu) vulnerabilities. There is a component openssl (3.0.2-0ubuntu1.6) for which Dependency Track has found two critical vulnerabilities and also log4j (1.2.16) with two critical vulnerabilities.

Our policy's conditions look like this: Severity is Critical PURL does not match pkg:deb

However, only openssl is raised (wrongly) by the policy as violation. This is because the severity evaluator returns two violations and PURL evaluator returns zero violations. Thus, the number of violations and conditions match. log4j is not raised as a violation because severity evaluator returns two violations and PURL evaluator also returns 1 violation making the total as three which is not equal to number of conditions.

I think there are two possible ways to fix this:

1. Modify every evaluator to return at most one violation
2. Modify the condition in branch for All operator to check that there is at least one violation for each condition

We are using v4.5.0 of Dependency Track