@tracerbench/tracerbench-compare-action

GitHub Action for TracerBench https://www.tracerbench.com/

What is it?

GitHub Action CI setup for TracerBench Compare command; finely tuned for benchmarking Ember applications and Addons. Please read the consumed TracerBench Compare command for associated configuration and API details: https://www.tracerbench.com/docs/api/compare

Usage as a GitHub Action

You can use this action by adding it to an existing workflow or creating a new workflow in your project.

For example, the below adds a check for the users route to all pull requests to the master branch.

name: PerformanceCheck

on:
  pull_request:
    branches:
      - master

jobs:
  analyze-users-route:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v1
        with:
          fetch-depth: 0
      - uses: tracerbench/tracerbench-compare-action@master
        with:
          experiment-url: 'http://localhost:4201/users'
          control-url: 'http://localhost:4200/users'
          markers: 'end-users-model-load'
          regression-threshold: 25
          fidelity: high

Local Usage and Usage with other CI Systems

The GitHub Action project provides a small wrapper that pipes the configuration into the project's main. This allows for easy use in any setup (local or CI) by adding this action as a dependency.

yarn add @tracerbench/tracerbench-compare-action

For example, you could mirror the above check in an Ember Application by doing the following.

// perf-test-config.json

{
  "experiment-url": "http://localhost:4201/users",
  "control-url": "http://localhost:4200/users",
  "markers": "end-users-model-load",
  "regression-threshold": 25,
  "fidelity": "high"
}

// perf-test.js

const analyze = require('@tracerbench/tracerbench-compare-action');
const config = require('./perf-test-config.json');

analyze(config);

Configuration Options