distributed icon indicating copy to clipboard operation
distributed copied to clipboard

Adjust popen timeouts for testing

Open fjetter opened this issue 1 year ago • 2 comments

This popen utility is often used in our CLI tests. The timeouts here are however not well adjusted to our overall test timeouts. Particularly if this is used inside of an async gen_test / gen_cluster test these timeouts do not match since the gen_cluster default test timeout is 30s.

Therefore, the async test timeout of 30s will always win over this popen timeout but timeouts should be hierarchical since they otherwise overshadow problems. The test test_nanny_worker_port_range_too_many_workers_raises for instance is failing intermittently on CI but there is no way to tell where or why it is stuck because the async timeout hits before anything else is happening.

Generally, 30s and 10s for a process termination / kill is way too generous so I suggest to dial both of them down a bit

fjetter avatar Aug 30 '24 15:08 fjetter

Same goes for distributed.cli.tests.test_dask_spec.test_errors. Since the asyncio timeout hits we don't even get the subprocess output

fjetter avatar Aug 30 '24 15:08 fjetter

Unit Test Results

See test report for an extended history of previous test failures. This is useful for diagnosing flaky tests.

    25 files  ±0      25 suites  ±0   10h 25m 5s :stopwatch: + 8m 12s  4 129 tests ±0   4 009 :white_check_mark:  -  9    110 :zzz: ±0  10 :x: + 9  47 698 runs  ±0  45 589 :white_check_mark:  - 13  2 095 :zzz: ±0  14 :x: +13 

For more details on these failures, see this check.

Results for commit d1f50133. ± Comparison against base commit 36020d6a.

:recycle: This comment has been updated with latest results.

github-actions[bot] avatar Aug 30 '24 16:08 github-actions[bot]