[bug] I suspect the --failOnConsole parameter doesn't work any more.

[work933k]

Describe the bug I run test-storybook with the --failOnConsole parameter, but the tests pass even while there are errors in the browser-console.

To Reproduce test-storybook --url http://localhost:6006 --index-json --failOnConsole --config-dir libs/dummy-app/shared/choose-product-date/.storybook

@storybook/[email protected]

Output

Running command: test-storybook --url http://localhost:6006 --index-json --failOnConsole   --config-dir libs/dummy-app/shared/choose-product-date/.storybook
Serving "dist/storybook/dummy-app-shared-choose-product-date" at "http://localhost:6006/"
[test-storybook] Cleaning up /tmp/86f0791b657a5727a17427b83e310a61
 PASS   browser: chromium  ../../tmp/86f0791b657a5727a17427b83e310a61/pensionpostponecomponent.test.js
 PASS   browser: chromium  ../../tmp/86f0791b657a5727a17427b83e310a61/participationdetailcomponent.test.js
 PASS   browser: chromium  ../../tmp/86f0791b657a5727a17427b83e310a61/pensionbringforwardcomponent.test.js
 PASS   browser: chromium  ../../tmp/86f0791b657a5727a17427b83e310a61/choosepensiondatecomponent.test.js
 PASS   browser: chromium  ../../tmp/86f0791b657a5727a17427b83e310a61/participationapplychoicescomponent.test.js

Test Suites: 5 passed, 5 total
Tests:       5 passed, 5 total
Snapshots:   0 total
Time:        5.682 s
Ran all test suites.

It used to work before.

[matttk]

We also ran into this yesterday. Tried console.log in various places and it never works anywhere.

[yannbf]

Hey there @matttk @work933k could you provide a reproduction? I just tested the flag in two different projects and it works as intended. The flag is only intended to fail on console.error calls during the rendering or play function, btw.

[sunilrathore24]

@yannbf correct me if i am wrong, in my angular application i am getting console error regarding missing component since i have not added related module, but when i run yarn test-storybook --url http://localhost:4400 --config-dir apps\storybook\.storybook --failOnConsole -u --includeTags="checkbox" the test is passing with out any error, idealy it should fail right? version - "@storybook/test-runner": "^0.18.2",