rules_closure
rules_closure copied to clipboard
Published
20 hours ago •
bazelbuild
bazelbuild
closure_js_test - Passed tests sometimes hang
We are just starting to use rules_closure for real, and the second test suite that I've migrated shows flaky behavior on our CI system. I was hoping for some advice on how to diagnose this.
We use teamcity, and when running the suite 3 times in a row, one test finished in TIMEOUT the first run, a different test the second, and all tests passed on the third. It ran on 3 different hosts, but they are identically configured with ansible.
The tests don't take long to run, and the runner outputs "all tests passed" long before the timeout. They do not do anything that could account for a long delay.
First run:
[13:16:46] ==================== Test output for //js/corp/consumerfeedbackstorm/invites/templates:addtemplatedialog_test:
[13:16:46] Listening http://localhost:35355/index.html
[13:16:46] Serving /index.html
[13:16:46] Serving /filez/corp/js/corp/consumerfeedbackstorm/invites/templates/addtemplatedialog_test_bin.js
[13:16:46] Serving /filez/corp/js/vendor/jquery-ui-1.8.24.min.js
[13:16:46] Serving /filez/corp/js/vendor/jquery-1.10.2.min.js
[13:16:46] -> 13:11:33.964 : Starting tests: Untitled Test Case
[13:16:46] -> 13:11:33.965 : Running test: testErrorRendering
[13:16:46] -> 13:11:33.975 : testErrorRendering : PASSED
[13:16:46] -> 13:11:33.975 : Running test: testUserInput
[13:16:46] -> 13:11:34.003 : testUserInput : PASSED
[13:16:46] -> 13:11:34.003 : 2 of 2 tests run in 38.78055200000017ms.
[13:16:46] -> 2 passed, 0 failed.
[13:16:46] 19 ms/test. 4 files loaded.
[13:16:46] -> 13:11:34.003 : Tests complete
[13:16:46] Terminated
[13:16:46] -- Test timed out at 2019-07-21 13:16:30 EDT --
[13:16:46] ================================================================================
[13:16:46] //js/corp/consumerfeedbackstorm/accountsettings:customlegal_test PASSED in 1.7s
[13:16:46] //js/corp/consumerfeedbackstorm/accountsettings:emailinvites_test PASSED in 1.8s
[13:16:46] //js/corp/consumerfeedbackstorm/accountsettings:maxcontactfrequency_test PASSED in 2.3s
[13:16:46] //js/corp/consumerfeedbackstorm/accountsettings:quarantine_test PASSED in 1.5s
[13:16:46] //js/corp/consumerfeedbackstorm/accountsettings:settingsrow_test PASSED in 1.8s
[13:16:46] //js/corp/consumerfeedbackstorm/accountsettings:surveypageenabled_test PASSED in 1.3s
[13:16:46] //js/corp/consumerfeedbackstorm/accountsettings:textinvitesettingsvalidator_test PASSED in 0.8s
[13:16:46] //js/corp/consumerfeedbackstorm/accountsettings:textinvitesettingsview_test PASSED in 1.9s
[13:16:46] //js/corp/consumerfeedbackstorm/invites:reviewinvitationvalidator_test PASSED in 1.2s
[13:16:46] //js/corp/consumerfeedbackstorm/invites/templates:templatevalidator_test PASSED in 1.1s
[13:16:46] //test/com/corp/consumerfeedback:unit PASSED in 48.1s
[13:16:46] //js/corp/consumerfeedbackstorm/invites/templates:addtemplatedialog_test TIMEOUT in 315.1s
[13:16:46] /home/cdbuilder/.cache/bazel/_bazel_TeamCity/df2b12f83f809abace199d393a19f0d3/execroot/corp/bazel-out/k8-fastbuild/testlogs/js/corp/consumerfeedbackstorm/invites/templates/addtemplatedialog_test/test.log
[13:16:46]
[13:16:46] Executed 12 out of 12 tests: 11 tests pass and 1 fails locally.
[13:16:46]
Second run
[14:46:17] ==================== Test output for //js/corp/consumerfeedbackstorm/accountsettings:settingsrow_test:
[14:46:17] Listening http://localhost:54287/index.html
[14:46:17] Serving /index.html
[14:46:17] Serving /filez/corp/js/corp/consumerfeedbackstorm/accountsettings/settingsrow_test_bin.js
[14:46:17] Serving /filez/corp/js/vendor/jquery-ui-1.8.24.min.js
Serving /filez/corp/js/vendor/jquery-1.10.2.min.js
[14:46:17] -> 14:41:04.476 : Starting tests: Untitled Test Case
[14:46:17] -> 14:41:04.476 : Running test: testCancelReset
[14:46:17] -> 14:41:04.497 : testCancelReset : PASSED
[14:46:17] -> 14:41:04.497 : Running test: testRenderErrors
[14:46:17] -> 14:41:04.500 : testRenderErrors : PASSED
[14:46:17] -> 14:41:04.500 : Running test: testSubmitToggle
[14:46:17] -> 14:41:04.502 : testSubmitToggle : PASSED
[14:46:17] -> 14:41:04.502 : Running test: testToggleEdit
[14:46:17] -> 14:41:04.513 : testToggleEdit : PASSED
[14:46:17] -> 14:41:04.514 : 4 of 4 tests run in 38.01679799999988ms.
[14:46:17] -> 4 passed, 0 failed.
[14:46:17] 10 ms/test. 4 files loaded.
[14:46:17] Loading:
[14:46:17] -> 14:41:04.514 : Tests complete
[14:46:17] Terminated
[14:46:17] Loading: 0 packages loaded
[14:46:17] -- Test timed out at 2019-07-21 14:46:02 EDT --
[14:46:17] ================================================================================
[14:46:17] //js/corp/consumerfeedbackstorm/accountsettings:customlegal_test PASSED in 0.9s
[14:46:17] //js/corp/consumerfeedbackstorm/accountsettings:emailinvites_test PASSED in 1.0s
[14:46:17] //js/corp/consumerfeedbackstorm/accountsettings:maxcontactfrequency_test PASSED in 1.1s
[14:46:17] //js/corp/consumerfeedbackstorm/accountsettings:quarantine_test PASSED in 1.1s
[14:46:17] //js/corp/consumerfeedbackstorm/accountsettings:surveypageenabled_test PASSED in 1.1s
[14:46:17] //js/corp/consumerfeedbackstorm/accountsettings:textinvitesettingsvalidator_test PASSED in 1.1s
[14:46:17] //js/corp/consumerfeedbackstorm/accountsettings:textinvitesettingsview_test PASSED in 1.2s
[14:46:17] //js/corp/consumerfeedbackstorm/invites:reviewinvitationvalidator_test PASSED in 0.9s
[14:46:17] //js/corp/consumerfeedbackstorm/invites/templates:addtemplatedialog_test PASSED in 0.9s
[14:46:17] //js/corp/consumerfeedbackstorm/invites/templates:templatevalidator_test PASSED in 0.9s
[14:46:17] //test/com/corp/consumerfeedback:unit PASSED in 19.2s
[14:46:17] //js/corp/consumerfeedbackstorm/accountsettings:settingsrow_test TIMEOUT in 315.0s
Third run didn't print anything because it was successful, taking 1m44s to build/run the java and js test suites.
This seems very much like the JS test suites succeeds, but for whatever reason Bazel doesn't get that memo and waits until the 5 minute limit expires, after which Bazel declares the test timed out.
Have you seen anything like this before?
