eclipse.platform icon indicating copy to clipboard operation
eclipse.platform copied to clipboard

Add a watcher (JUnit 3) test rule (JUnit 4) for hanging test logging

Open HeikoKlare opened this issue 2 years ago • 1 comments

Tests may be (and actually are every now and then) hanging for different reasons, such as livelocks or deadlocks. A hanging test does not log anything on its own anymore, which makes it difficult to identify a hanging test and the reasons for that. To this end, it would be useful to have an additional entity that logs the state of all threads, i.e., a thread dump, when a test takes longer than a defined threshold.

Since several tests need to run in the UI thread, the JUnit Timeout rule cannot be used, as it executes a test in a different thread that can even be terminated preemptively after a timeout. Instead, this change introduces a HangingTestRule that spawns a watcher thread that writes a thread dump whenever a test (that is run in the UI thread) takes more time than a defined timeout. In total, this change does the following:

  • Makes the thread dump logic currently placed in TestBarrier2 reusable in a central TestUtil
  • Adds a HangingTestWatcher that writes a thread dump if not stopped after a given timeout (compatible with JUnit 3)
  • Adds a HangingTestRule that wraps the HangingTestWatcher into a JUnit 4 rule
  • Adds the HangingTestWatcher to the JUnit 3 CoreTest class to apply the logging to all subclasses of CoreTest

Also see https://github.com/eclipse-platform/eclipse.platform.ui/issues/1229, which discusses some additional logging in Platform UI to help debugging hanging tests. That may be replaced by the proposed watcher/rule.

HeikoKlare avatar Oct 26 '23 17:10 HeikoKlare

Test Results

     534 files  ±0       534 suites  ±0   1h 11m 13s :stopwatch: + 8m 5s   3 793 tests ±0    3 787 :heavy_check_mark:  - 1    5 :zzz: ±0  1 :x: +1  12 000 runs  ±0  11 963 :heavy_check_mark:  - 1  36 :zzz: ±0  1 :x: +1 

For more details on these failures, see this check.

Results for commit 1dd26d02. ± Comparison against base commit d0ec0a8a.

:recycle: This comment has been updated with latest results.

github-actions[bot] avatar Oct 26 '23 18:10 github-actions[bot]

What is the status of this one?

akurtakov avatar Dec 17 '24 08:12 akurtakov

This implementation was made for JUnit 3/4 and we have migrated most tests to JUnit 5 by now. So I would need to adapt the PR to enable the functionality as a JUnit 5 extension. Since I have not seen hanging tests for quite a while, I would currently say it's not worth it. If someone thinks differently, let me know and I can try to update the PR. For now, I will close it.

HeikoKlare avatar Jan 02 '25 16:01 HeikoKlare