deno_std icon indicating copy to clipboard operation
deno_std copied to clipboard

--fail-fast doesn't work with describe/test

Open dandv opened this issue 11 months ago • 2 comments

Describe the bug

Passing --fail-fast to deno test is documented to stop on the first failure. This works with Deno.test(), but not with describe / expect.

Steps to Reproduce

main_test.ts:

import { describe, test } from '@std/testing/bdd';
import { assertEquals } from "@std/assert";

describe('addition', () => {
  test(function addTest1() {
    assertEquals(1 + 1, 42);
  });

  test(function addTest2() {
    assertEquals(2 + 2, 4);
  });
});
$ deno test --fail-fast
Check file:///home/dandv/deno-bugs/main_test.ts
running 1 test from ./main_test.ts
addition ...
  addTest1 ... FAILED (7ms)
  addTest2 ... ok (1ms)
addition ... FAILED (due to 1 failed step) (12ms)

 ERRORS

addition ... addTest1 => https://jsr.io/@std/testing/1.0.9/_test_suite.ts:393:15
error: AssertionError: Values are not equal.
[...]

Expected behavior

addTest2 should not have executed.

Environment

  • OS: Fedora Linux 38
  • deno version: 2.1.4
  • std version: @std/testing@^1.0.9

dandv avatar Jan 15 '25 13:01 dandv

@kt3k: should this be an enhancement or a bug? If it really should be an enhancement, then the docs for describe and test should mention they don't support --fail-fast, otherwise developers may assume they do, since it's a deno argument.

dandv avatar Feb 18 '25 14:02 dandv

--fail-fast currently only works for the top-level describe and test (The below test stops at suite 1, for example). So the issue here is --fail-fast doesn't work for nested test cases.

import { describe, test } from '@std/testing/bdd';
import { assertEquals } from "@std/assert";

describe("suite 1", () => {
  test(function addTest1() {
    assertEquals(1 + 1, 42);
  });
});


describe("suite 2", () => {
  test(function addTest2() {
    assertEquals(2 + 2, 4);
  });
});

kt3k avatar Feb 19 '25 04:02 kt3k