Fork opitmised for pytest-benchmark (upstream improvements?)

[chrisjsewell]

Hey, thanks for the great package! There was a number of additional data aspects I wanted to capture/utilise from pytest-benchmark (in particular test grouping) and so I created a fork optimised for this: https://github.com/aiidateam/github-action-benchmark, and you can see an example output here: https://aiidateam.github.io/aiida-core/dev/bench/ubuntu-18.04/django/

Now that I've just about finalised my implementation, I thought it would be good to feedback, since I sure at least some of this can be upstreamed to here 😄

I've outlined the changes in the README (and also the commit history is pretty readable), but I'll copy it here:

1. More metadata is saved about the benchmark run, for later comparison:

   • CPU data (cores, processors, speed) is specifically saved, per commit, in the cpu key (uses systeminformation npm package).
   • The python version is extracted from the pytest-benchmark data in an extra key
   • A metadata action option is available to save additional data
   • This data is all shown in the data point's tooltip

2. The group name is saved for each test (see pytest-benchmark markers), or null if no group is given. In the web-page rendering, tests are then arranged by the group they are in, which can be given a sub-heading, description, etc, and also a group can be "consolidated" into a single chart (with handling of differing data points).

3. A new render-json-path allows for a JSON file to be copied, which is used to configure the rendered web-page, e.g. for certain test suites and groups:

    {
        "suites": {
            "name": {
                "header": "Test Suite Title",
                "description": "Description of test suite."
            }
        },
        "groups": {
            "group1": {
                "header": "Group 1 Title",
                "description": "Description of group 1.",
                "single_chart": true,
                "xAxis": "date",
                "backgroundFill": false
            }
        }
    }
   

4. Split original index.html into multiple HTML/JS files in the src/assets folder. This allows for easier testing and development of the output web-page. This has additionally allowed for:

   • Adding npm run serve, for local development of output web-page (using light-server)
   • Adding overwrite-assets option, to specify whether any existing assets should be overridden during a commit to gh-pages.

5. Improve formatting of charts:

   • Color cycling for consolidated charts
   • For legend, extract common test name prefix as title
   • Data point tooltips: rounding values to 5 significant figures and better formatting of dates etc.
   • Addition of the chartjs-plugin-zoom and a "Reset Zoom" button.

6. Add commit-msg-append option , useful for adding e.g. [ci skip] to commit message, but not having it as part of the test suite key in the data JSON.

7. Removed capture of commit author/committer, since it can be obtained from the commit id/url, and just bloats the data JSON.

8. Renamed max-items-in-chart -> max-data-items, to better describe its function of truncating the saved data during a commit.