`deno test` exits prematurely if a test empties the event loop

[h4l]

If a test run via Deno.test() allows the event loop to empty, the test suite immediately fails without reporting the result of the test or any subsequent tests.

The failure message is also somewhat confusing. It seems to be technically accurate, and makes sense if you understand what's happened. But confusing when you don't realise you've accidentally emptied the event loop, allowing it to halt and are trying to work out why your test is failing (actually not executing).

As a user of the test API, I'd expect in this situation that the test would hang until a timeout was triggered to fail the test.

Maybe an argument for tests having timeouts (per #11133)? E.g. if the test runner had a pending timer for the timeout, the event loop couldn't become empty while a test was executing.

This test module demonstrates the problem:

// empty_event_loop_test.ts
import { assertEquals } from "https://deno.land/[email protected]/testing/asserts.ts";

function someApi(flag: boolean) {
  return new Promise((resolve) => {
    if (flag) {
      resolve("foo");
    } else if (!"bug") {
      resolve("bar");
    } else {
      console.log("* accidentally not resolving promise");
    }
  });
}

Deno.test("someApi returns foo when flag is true", async () => {
  const result = await someApi(true);
  assertEquals(result, "foo");
});

Deno.test(
  "[premature-exit]: deno test terminates the whole suite when a test empties the event loop",
  async () => {
    // event loop exits while awaiting someApi's promise
    console.log("\n\nBefore await");
    const result = await someApi(false);
    // this never executes because result is never resolved
    console.log("After await");
    assertEquals(result, "bar");
  },
);

Deno.test("[premature-exit]: simplified example", () => {
  return new Promise(() => {});
});

Deno.test("This is never executes if a premature-exit example does", () => {});

The tests stop executing when the first [premature-exit] test runs:

$ deno test empty_event_loop_test.ts
running 4 tests from file:///private/tmp/empty_event_loop_test.ts
test someApi returns foo when flag is true ... ok (8ms)
test [premature-exit]: deno test terminates the whole suite when a test empties the event loop ...

Before await

* accidentally not resolving promise


test result: FAILED. 1 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out (35ms)

error: Promise resolution is still pending but the event loop has already resolved.

Final test executes as expected if no [premature-exit] tests run:

$ deno test empty_event_loop_test.ts --filter '/^someApi|never executes/'
running 2 tests from file:///private/tmp/empty_event_loop_test.ts
test someApi returns foo when flag is true ... ok (10ms)
test This is never executes if a premature-exit example does ... ok (6ms)

test result: ok. 2 passed; 0 failed; 0 ignored; 0 measured; 2 filtered out (164ms)

The simplified [premature-exit] also causes a premature exit:

$ deno test empty_event_loop_test.ts --filter '/\[premature-exit\]: simplified|never executes/'
running 2 tests from file:///private/tmp/empty_event_loop_test.ts
test [premature-exit]: simplified example ...
test result: FAILED. 0 passed; 0 failed; 0 ignored; 0 measured; 2 filtered out (30ms)

error: Promise resolution is still pending but the event loop has already resolved.

[stale[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed in 7 days if no further activity occurs. Thank you for your contributions.

[h4l]

It's still an issue, but should be fixed by the work in #11133.

[krvajal]

Any idea when this will be fixed. It is two years old by now. I am happy to help as well

[bartlomieju]

@mmastrac can you take a look when you find time? Seems related to your sanitizer work

[skybrian]

For people who searched for this bug, a workaround is to set your own timeout using setTimer:

  beforeEach(() => {
    setTimeout(() => {}, 1000);
  });

(Perhaps failing with a better error message.)

In my case, as soon as I did this, the test hung until it timed out, and I realized that a bug in the test was causing a deadlock.