clusterfuzz icon indicating copy to clipboard operation
clusterfuzz copied to clipboard

Implement grouping test cases by culprit commit

Open mi-ac opened this issue 2 years ago • 1 comments

There are currently three grouper options: _group_testcases_with_similar_states(testcase_map) _group_testcases_with_same_issues(testcase_map) _group_testcases_based_on_variants(testcase_map)

Additionally to those, for some projects (with exact revision resolution) like V8, it would be beneficial to group also by regression range (when it points to a single commit). This would prevent bug floods of correctness fuzzing like https://crbug.com/1443672 and might be an alternative to a generic issue-filing limit (https://github.com/google/clusterfuzz/issues/2706).

There might be jobs or crash types though for which such grouping renders a problem? Haven't thought it through yet. Hence, such a feature might need to be "opt-in" per job or have an exclude list for some jobs or crash types. E.g. a commit that enables a new sanitizer option might have several distinct bugs bisecting back to it...

For V8 correctness fuzzing though, it would certainly be a big plus to group by suspected commit.

mi-ac avatar May 11 '23 20:05 mi-ac

I strongly support this idea, and have had similar thoughts on this for a while.

jonathanmetzman avatar May 23 '23 20:05 jonathanmetzman