trufflehog
trufflehog copied to clipboard
Fix nondeterminism in custom detectors
Previously, a custom detector configured to use multiple patterns would produce its results from a match in a nondeterministic order. This behavior could be seen by running the custom detectors test in a loop, e.g.,
$ cd pkg/custom_detectors
$ while true; do go test; done
This is likely to reveal sporadic test failures.
This nondeterminism was a result of producing results by iterating over a map, which has unspecified order in Go.
The fix: sort the intermediate map data structure prior to producing results.
The failures were sporadically seen in GitHub Actions jobs as well. For example:
- https://github.com/trufflesecurity/trufflehog/actions/runs/17478249783/job/49643183088
- https://github.com/trufflesecurity/trufflehog/actions/runs/17459343592/job/49580167352
- https://github.com/trufflesecurity/trufflehog/actions/runs/17459412883/job/49580376149
- https://github.com/trufflesecurity/trufflehog/actions/runs/17466455637/job/49603364126
Note, there is an earlier PR (#4438) to address the test flakiness here. The approach taken there is different: it explicitly allows the two permutations of the test, rather than changing the detector to eliminate the nondeterminism.
Also, I'll point out here that the fix for the nondeterminism in this PR (eliminating it from the detector) will fix the test flakiness, making the order of the Raw result output consistent from run to run. However, the consistent order will still be rather unpredictable to users, as it's based on the matched content, and not based on something like the name of the custom regexes supplied.