Create summary signature hash for benchmark tests without suite-level value.

[gmierz]

trafficstars

This patch creates a summary signature hash for benchmark tests without a suite-level value to prepare for moving away from using a suite-level value. Instead, the suite-level value will be stored as a subtest with replicates. With this change, removing the value from a benchmark won't reset it's signature anymore.

[gmierz]

@beatrice-acasandrei I'm looking at starting to remove some suite-level values for benchmark tests (speedometer3 atm), but an issue we're hitting is that removing the value resets the signature/series for the subtests.

What do you think about this approach for resolving that issue? An alternative is we remove the summary signature hash from the subtest signature creation but this would reset all of the existing signatures that already use a summary signature hash.

[codecov-commenter]

Codecov Report

All modified and coverable lines are covered by tests :white_check_mark:

Comparison is base (5a4df58) 77.06% compared to head (7208818) 77.05%.

Additional details and impacted files

@@            Coverage Diff             @@
##           master    #7889      +/-   ##
==========================================
- Coverage   77.06%   77.05%   -0.02%     
==========================================
  Files         541      541              
  Lines       26794    26795       +1     
  Branches     3364     3364              
==========================================
- Hits        20650    20648       -2     
- Misses       5977     5980       +3     
  Partials      167      167              

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.

[beatrice-acasandrei]

Hi @gmierz! Because I haven't worked myself on this part of Treeherder I have some questions, hopefully the questions are not that bad. Thanks.