Thresholds evaluated incorrectly

[ram-gupta]

trafficstars

Brief summary

When test script has errors thresholds is evaluated incorrectly indicating success instead of failure.

k6 version

0.49

OS

macOS 14.6

Docker version and image (if applicable)

No response

Steps to reproduce the problem

Run test with below script:

import { check } from "k6";
import http from "k6/http";
export const options = {
  stages: [
    { duration: "2s", target: 1 }
  ],
  thresholds: {
    "checks": [{ threshold: "rate>=0.9", abortOnFail: true }]
  }
}


export default function (setupData) {

  const result = http.get("https://test-api.k6.io/public/crocodiles/");
  throw new Error("This is an error");

  check(result, {
    "status:200": result.status === 200,
  });
}

Test is not failed even though checks metric value is 0. The summary json also indicates same:

"checks": {
      "type": "rate",
      "contains": "default",
      "values": {
        "passes": 0,
        "fails": 0,
        "rate": 0
      },
      "thresholds": {
        "rate>=0.9": {
          "ok": true
        }
      }
    },

Expected behaviour

Test should fail when thresholds is breached even if test script has errors.

Actual behaviour

Test is passed even though thresholds is breached.

[Fernando-hub527]

Hello, I would like to work on this issue!

[arpitaso]

Hello @Fernando-hub527 any update on this issue? We observed that k6 released latest version 57,1.0 in any of these 2 versions is the above issue fixed?

[Fernando-hub527]

Hey @arpitaso , sorry for the delay. Just opened the PR.

[arpitaso]

Hey @Fernando-hub527 , would you mind helping with the merge of this PR? Thanks in advance!