best icon indicating copy to clipboard operation
best copied to clipboard

Feature request: ability to measure benchmark deltas across commits/branches

Open nolanlawson opened this issue 6 years ago • 1 comments

Use case: Locker Service would like to measure the performance of the browser eval() vs Locker secureEval(). We can write Best benchmarks to independently measure each, but there's no way AFAICT to measure the delta between the two, across commits.

So for instance, if eval() itself regresses between Chrome 71 and Chrome 72, we don't care. But if the delta between our secureEval() and eval() regresses, then we actually care. We can track the two independently and infer this, but it would be better if Best had something built-in to handle this use case.

Other use cases that may want this scenario: measuring LWC vs Aura, polyfill vs native, etc.

nolanlawson avatar Jan 25 '19 19:01 nolanlawson

Not sure I understand this use case. Let's talk

diervo avatar Feb 01 '19 15:02 diervo