codespeed
codespeed copied to clipboard
tobami
changes: add revision-vs-revision comparison option
This is a feature request.
Description.
New view option for "Changes" page. In addition to "show n last" revisions, it makes possible to select two revisions by their commit hash. It's not obvious what to do with "trend" column, but "changes" column will show difference between two revisions (better/worse).
==
The required functionality is "revision vs revision" comparison (same executable, same environment, ...). If codespeed already has one, which escaped my attention, this feature request can be closed.
The closest solution I found is to create two new executables, each of them includes results of the interest. Then "Comparison" page can be used to compare them. It could require only one temporary executable if there was an option to select the revision.
But for some reason, this leads to empty comparison plot (no bars, see image below).
The big picture: I want to compare some older version with newer (say, year later release) and use benchmarks with geomean to compare them. I'm OK with any representation: be it +/- change like in "Changes" or bar plot like in "Comparison".
The empty plots are a consequence of the absence of lastest commit in those executable results. Not sure if this is an expected behavior (from functional point of view).
By changing results revision to the default branch tip, plots are no longer empty:
So is the feature request to make it possible to compare any two commit hashes?
Yes. I will come back with reasons why this is useful.
IIRC it's useful when you know exact hashes when feature A was enabled and when it was disabled. You want to compare those to measure the effects.