Multihttp & Scripted check assertions - success rate by probe graph doesn't show failures properly

[peterschretlen]

What happened: Assertions in a multihttp and/or scripted check are failing, but the chart does not show when the failures occur.

Here is an example of an assertion that failed 58 times over the course of a week. The success chart shows some gaps and some dots, but it never goes below 100%. This makes it very hard to narrow down when/where the failures occurred.

What you expected to happen:

If I open explore from the dashboard panel I can see that the interval is every 2m, for a check that runs on a 2m interval. When I change the interval to 5m, I can now see where the failures occured

How to reproduce it (as minimally and precisely as possible):

This script will randomly fail about 10% of the time. Run this as a scripted check, and look at the assertion graph for '2. is result < 10'. The plot lines will never go below 100% even though it is failing.

import { check, fail } from 'k6'
import http from 'k6/http'

export default function main() {
  const res = http.get('https://www.randomnumberapi.com/api/v1.0/random?min=1&max=10');
  // console.log will be represented as logs in Loki
  console.log(`response ${res.json('[0]')}, ${res.status}`);
  check(res, {
    '1. is status 200': (r) => r.status === 200,
    '2. is result < 9': (r) => parseInt(res.json('[0]')) <  9,
  }) || fail(`check failed`);;
}