Compare traces on span durations

[tiffon]

Requirement - what kind of business use case are you trying to solve?

Facilitate the analysis of latency between two traces. E.g. trace comparisons on span durations instead of only structural aspects.

Problem - what in Jaeger blocks you from solving the requirement?

Functionality is not present.

Proposal - what do you suggest to solve the problem or improve the existing situation?

Proof of concept in the diff-on-latency branch.

Compare two traces on the average duration of spans within a node in the comparison tree.

Structural comparison of two traces:

Comparison of durations:

POC has toggles for using a square-root scale and relative diff comparisons (e.g. comparing on percentage difference vs time).

Any open questions to address

Baseline for the comparison

max(a, b)

Currently, the comparison is:

(b - a) / max(a, b) * 100

An example where the average duration of spans in the node from trace A is larger:

A: 780ms
B: 470ms
Diff: -309ms (-40%)

Conversely, an example where the avg duration of spans in the node from trace B is larger:

A: 447ms
B: 2,265ms
Diff: +1,820ms (+80%)

a

Alternatively, if we choose a as the baseline, the equation become:

(b - a) / a * 100

The scenarios become:

A: 780ms
B: 470ms
Diff: -309ms (-40%)

Conversely, an example where the avg duration of spans in the node from trace B is larger:

A: 447ms
B: 2,265ms
Diff: +1,820ms (+407%)

[yurishkuro]

(b - a) / max(a, b) * 100 doesn't make a lot of sense to me, how could we describe the semantics of that?

(b - a) / a * 100 is the normal delta (after vs. before) as percentage, makes more sense.

Are the color gradient endpoints just matched to the min/max deltas across the whole trace?

What are the action items to get this branch merged, as experimental view?

Would it make sense to apply borders to the nodes to increase contrast?

[tiffon]

(b - a) / a * 100 is the normal delta (after vs. before) as percentage, makes more sense.

SGTM.

Are the color gradient endpoints just matched to the min/max deltas across the whole trace?

There is a relative difference mode, but otherwise, the coloring is based on the maximum absolute difference across the trace. The coloring uses a square-root scale to prevent relevant values from being washed-out.

What are the action items to get this branch merged, as experimental view?

The branch will have a lot of conflicts with trace graphs. So, I'd like to get that PR in, first. Then, I'll deal with the conflicts. The POC was also implemented in a hurry, so it needs quite a bit of cleanup, too.

Would it make sense to apply borders to the nodes to increase contrast?

SGTM.

[Max-Sum]

This seems to be promising! Do you have a schedule on merging?

[bobrik]

I'm very interested in having this. Should I look into reviving @tiffon's efforts myself or is this planned already?

[yurishkuro]

@bobrik @tiffon no longer has time to work on this, and we don't have anyone else. I agree this is a useful feature.