lighthouse-ci icon indicating copy to clipboard operation
lighthouse-ci copied to clipboard

Track which report generated each summary statistic in db

Open minimorph223 opened this issue 4 years ago • 3 comments

Describe the bug the overall score displayed in dashboard page is different with the score displayed in detail report or compare page.

To Reproduce Steps to reproduce the behavior:

  1. collect statistics with numberOfRuns 3.
  2. go to dashboard, the score is '50'. image
  3. click the 'report'.
  4. the displayed score in report page is '51' image
  5. '50' is the median score number which is expected. '51' is the max number.

Environment (please complete the following information):

  • lhci server version: 0.8.0
  • lhci client version: 0.8.0

minimorph223 avatar Aug 17 '21 08:08 minimorph223

Thanks for filing @minimorph223!

This is working as intended. The individual report you see is the median report while the score you see on the dashboard is the median score.

The current infrastructure does not track which report in the set contained the summary statistic in question. If we ever wanted to reduce confusion here, that would need to change.

patrickhulce avatar Aug 17 '21 13:08 patrickhulce

@patrickhulce thanks for the clarification. what is a median report means? when i do lhci upload, i saw it uploaded all reports, not median report.

minimorph223 avatar Aug 18 '21 01:08 minimorph223

when i do lhci upload, i saw it uploaded all reports, not median report.

Correct it does which is how median scores are computed.

what is a median report means?

Any place in the UI where the user can open a report for a specific build, the "representative report" (a.k.a. the median report) is used. It's selected using a blend of heuristics that attempts to pick the report that is most representative of all runs of the page across all metrics/scores. This is why the median report can be different from the median of specific metrics or category scores.

patrickhulce avatar Aug 18 '21 14:08 patrickhulce