inferno icon indicating copy to clipboard operation
inferno copied to clipboard

Differential output tooltips are confusing

Open itamarst opened this issue 1 year ago • 4 comments

Consider these two compressed line profiling reports.

Baseline:

base_only 50
both1 25
both2 25

Modified:

both1 25
both2 50
changed_only 125

The backwards looking difference flamegraph has explanatory tooltips as follows:

  • both1 (25 units, 12.50%; 0.00%)
  • both2 (50 units, 25.00%; +12.50%)
  • changed_only (125 units, 62.50%; +62.50%)

The forwards looking difference flamegraph:

  • base_only (50 units, 50.00%; -50.00%)
  • both1 (25 units, 25.00%; 0.00%)
  • both2 (25 units, 25.00%; +25.00%)

The main issue is the difference (+/-) percentages. They are really bad at explaining what happened.

Consider both2: it went from 25 to 50. In the backwards looking diff that's described as +12.50% even though it doubled. In the forwards looking diff that's described as both2 (25 other counts, 25.00%; +25.00%).

  1. This is inconsistent: +%12.50 vs +25.00%.
  2. It's talking about % of overall time, basically, but that's not what most people care about, they care about absolute change. "This function got faster" or "this function uses more memory", not "this function is higher percentage of the whole". If everything in your program somehow ran twice as slow, it'd be +0.00% even though everything is twice as slow!
  3. If % of total is really what you care about that's why there's a normalization option when creating the diff.

Consider changed_only: it didn't exist in the base profile, only in the changed profile. +62.50% does not convey that important piece of information, it just makes it seem like it grew 62% from the base.

itamarst avatar May 26 '23 16:05 itamarst