Report bad test() usage as a test failure, rather than a runtime error

[jamestalmage]

Sometimes you get sloppy / distracted and end up with a half completed test:

// you have some good tests:

test('foo', t => {
  // this will never be executed because of the mistakes below
});

// This throws - saying it should be a todo test:
test('fooo');

// Ooops - did you get distracted?:
test();

// Todo should not have an implementation, it is intended to document future tests/features.
test.todo('foo', t => {});

Instead of throwing when someone misuses the API, I think the misuse should just be reported as a failure. It should still execute the correctly written tests.

There is a $40.00 open bounty on this issue. Add more on Issuehunt.

• Checkout the Issuehunt explorer to discover more funded issues.
• Need some help from other developers? Add your repositories on Issuehunt to raise funds.

[sindresorhus]

I like the idea of being as graceful a possible. One concern though. Look at it the other way around. What if you have a lot of tests and you use one of them incorrectly. Now it will run all the tests before letting you know you have an issue and then you'll have to rerun the whole test file again. Just a thought.

[novemberborn]

#78 will likely force our hand here. We're doing #696 so we can correctly run exclusive test but I doubt we'll try and suss out these issues statically. If we don't then we'll end up in situations where we're already running a whole bunch of tests before the fault is encountered. Then why not run to completion?

[jamestalmage]

Now it will run all the tests before letting you know you have an issue and then you'll have to rerun the whole test file again. Just a thought.

No reason we can't report it as an error as soon as it's encountered.

[novemberborn]

No reason we can't report it as an error as soon as it's encountered.

You probably won't notice it though.

[jamestalmage]

Why not? If it increases the error count, that's noticeable immediately.

[novemberborn]

Why not? If it increases the error count, that's noticeable immediately.

Fair enough.

[novemberborn]

This will be easiest to implement once https://github.com/avajs/ava/pull/2217 lands.

[IssueHuntBot]

@issuehunt has funded $40.00 to this issue.

---

• Submit pull request via IssueHunt to receive this reward.
• Want to contribute? Chip in to this issue via IssueHunt.
• Checkout the IssueHunt Issue Explorer to see more funded issues.
• Need help from developers? Add your repository on IssueHunt to raise funds.