jest Ability to skip rest test cases if one fails

🚀 Feature Proposal

Provide ability to skip rest test cases in a file or describe block if one test fails.

Motivation

For E2E testing, the test cases are usually written to be run sequentially, and a failed test should stop the rest from running.

Example

test('should login', async () => {});

test('should edit profile', async () => {});

test('should render profile', async () => {});

Or with describes:

describe('#1', () => {
  test('should login', async () => {});

  describe('#1.1', () => {
    test('should edit profile', async () => {});

    test('should render profile', async () => {});
  });
});

Apr 27 '19 15:04 vilicvane

keyword: step prior art: https://github.com/rprieto/mocha-steps

Apr 27 '19 19:04 Mouvedia

Duplicate of https://github.com/facebook/jest/issues/6527 (for hooks and tests)

Apr 27 '19 21:04 jeysal

No it's not a duplicate. We are requesting mocha-steps not jasmine-fail-fast.

TL;DR: skip (and auto fail) the following steps after the first failing step within the current describe (not exit on first failure)

@jeysal please reopen.

Apr 27 '19 22:04 Mouvedia

Okay then, we can keep this open to track fail-fast within a describe block (although the OP does not explicitly request this). I think it's unlikely that this goes into Jest core though, while #6527 might as it's a logical extension of bail.

Apr 27 '19 22:04 jeysal

Personally I think this makes sense, and would love to see it in the jest core under step, as it lets you break tests into smaller parts (allowing you to describe them better, since they each have an it/step message), and increase performance for when you're testing something that is dependent on prior tests.

Take for example this test:

describe('.root', () => {
  it('contains the root package.json', () => {
    // eslint-disable-next-line @typescript-eslint/no-require-imports
    const requirePackageJson = () => require(path.join(pathTo.root, 'package.json'));

    expect(() => requirePackageJson()).not.toThrow();
    expect(requirePackageJson()).toStrictEqual(packageJson);
  });
});

It's checking that pathTo.root points to a directory with a package.json, and that the package.json is the actual root package.json (by comparing the object returned by the require to one imported via the normal module loader).

Since require throws an error if it can't find the file (the first expect), it makes no sense to test contents of the file (the second expect) if this happens.

But, since its are independent of each other, I either will have two tests failing (which while technically correct means I've got pointless noise that adds nothing to the situation), or one "big" it that combines two tests.

Meanwhile, if we had step:

describe('.root', () => {
  // eslint-disable-next-line @typescript-eslint/no-require-imports
  const requirePackageJson = () => require(path.join(pathTo.root, 'package.json'));

  step('has the root package.json', () => {
    expect(() => requirePackageJson()).not.toThrow();
  });
  step('contains the root package.json', () => {
    expect(requirePackageJson()).toStrictEqual(packageJson);
  });
});

This maybe be a result of me being relatively new to the testing game, but it seems to me that step would empower jest further, as it enables you to do feature-like testing letting you nicely test apps as a whole.

I know that step is pretty self explanatory in its advantages, I felt like it could be valuable to put one to paper - I've got a lot of tests that are similar to that.

I don't mind step not being in the jest core, just so long it's as easy to use as it, describe, #each, etc are today.

I'm just really excited at the idea of having this feature in jest, as I've come across this problem a lot! 😄

Apr 29 '19 01:04 G-Rath

Smoke tests in front of more substantial tests save people time. Without the ability to short circuit subsequent test, the time saving is mitigated.

Where does this functionality need to be added? Would this live in a test runner or is this a jest issue? (Many people don't even realize the distinction between jest and test runners...)

Jan 07 '20 03:01 cinderblock

Here is my temporary solution.

describe("test", () => {
  let skipOther = false
  test("check something", async () => {
    try {
      await checkSomething()
    } catch(e) {
      skipOther = true
      throw new Error("error")
    }
  })

  test("might be skipped", async () => {
    if(skipOther) throw new Error("error occured in previous tests.")
    console.log("display this, if there is no error before this.")
  })
})

Feb 25 '20 10:02 k1832

I would be interested in implementing a test.step function in jest circus, I don't see a way to do this outside of the jest-circus core though, as it requires adding some flag to the state.

May 01 '20 18:05 MrLoh

I created a PR that implements asynchronously canceling tests as described in #8604. With that in place, one could implement this functionality through easy monkey patching in user land like this:

let prevFailed;
const _describe = global.describe;
global.describe = (...args) => {
  prevFailed = false;
  return _describe(...args);
};
global.test.step = (name, fn, ...args) => {
  test(name, async function(...fnArgs) {
    if (prevFailed) {
      return this.skip();
    }
    try {
      return await fn(...fnArgs);
    } catch (e) {
      prevFailed = true;
      throw e;
    }
  }, ...args);
};

May 01 '20 22:05 MrLoh

@MrLoh where's the PR? I only see the issue

Sep 03 '20 09:09 FezVrasta

#9944 But I doubt it will ever be merged, I haven't heard back a single word from any contributor.

Sep 03 '20 11:09 MrLoh

My solution to skip rest tests after first failure https://github.com/facebook/jest/issues/6527#issuecomment-760092817

Jan 14 '21 11:01 davidglezz

Here is my solution #7245 to skip test programmatically and on run time after test began. the key is to call pending()

Mar 05 '21 09:03 ali-hellani

Playwright already provides a test.step https://playwright.dev/docs/api/class-test/#test-step And this would also make sense for jest specially for e2e tests.