[allure-jest] Provide an ability to mark test as `Status.BROKEN`

[maks-rafalko]

Is your feature request related to a problem? Please describe.

We need to have a way to highlight tests that are failing because of known bugs. Right now these tests are just failing and marked with red color.

However, the more tests we get - the harder it becomes to sort out unexpected failures in report.

On Jest level the perfect way to mark expected failures is test.failing('This test is failing and it's kind of ok now', ...) chainer.

But when we use it - such test has “passed“ status in Allure report - it's green. It's impossible to differentiate real passed tests from those that passed because we marked them with test.failing().

Describe the solution you'd like

Allure provides a static non-extendable set of statuses, and one of them is BROKEN: https://allurereport.org/docs/test-statuses/#broken

We would like to use it to mark test.failing() tests as BROKEN so they become in yellow color in Allure report. This way it's clear and immediately visible what tests are really passing, what tests are marked as test.failing(), thus "broken".

Describe alternatives you've considered Only prepend test case name with "FAILING: " or something like this prefix, but they still will be in green color. Not convenient.

Additional context

Currently, for Allure v2.15.1, we overwritten allure-jest/node jest environment, changing the handleTestPass() method to do the following:

- currentTest.status = Status.PASSED;
+ currentTest.status = testEntry.failing ? Status.BROKEN : Status.PASSED;

but it requires a lot of code and not developer-friendly solution, because we also need to copy getTestId and other functions, because they are not exported from allure-jest package

The full code:

// src/custom-allure-jest-node-environment.ts

import type {Circus} from "@jest/types";
// @ts-expect-error - module is available
import AllureJestEnvironment from "allure-jest/node";
import {Stage, Status} from "allure-js-commons";

export const getTestPath = (testEntry: Circus.TestEntry | Circus.DescribeBlock): string[] => {
  const path = [];
  let currentUnit: Circus.DescribeBlock | Circus.TestEntry | undefined = testEntry;

  while (currentUnit) {
    if (currentUnit.name) {
      path.unshift(currentUnit.name);
    }

    currentUnit = currentUnit.parent;
  }

  // first element is always ROOT_DESCRIBE_BLOCK, which shouldn't be reported
  return path.slice(1);
};

export const getTestId = (path: string[]): string => path.join(" ");

class CustomAllureEnvironment extends AllureJestEnvironment {
  handleTestPass(testEntry: Circus.TestEntry): void {
    const currentTestId = getTestId(getTestPath(testEntry));
    // @ts-expect-error - runningTests is private, but it's ok here
    const currentTest = this.runningTests.get(currentTestId)!;

    if (!currentTest) {
      // eslint-disable-next-line no-console
      console.error(`Can't find "${currentTestId}" test while tried to mark it as passed!`);
      return;
    }

    currentTest.stage = Stage.FINISHED;
    currentTest.status = testEntry.failing ? Status.BROKEN : Status.PASSED;
  }
}

export default CustomAllureEnvironment;

and then we changed jest env in jest.config.ts:

- testEnvironment: "allure-jest/node",
+ testEnvironment: "./src/custom-allure-jest-node-environment.ts",

---

So can we

• mark test.failing with Status.BROKEN status by default?
• or provide some way to conveniently work with this in user-land code