palfrey
rework timeouts so they are linked to waiting time of a test not total waiting time
Right now timeouts are counted starting from whatever point in time the test runner first picks them up. Having more cores then test can lead to all tests starting the timeout at the same time. With fewer cores they start after a (somewhat) random time.
This can be somewhat problematic as there is no clear and repeatable way to define how long timeouts actually are. a 60s timeout can mean "60s since the start of cargo test" or "60s since the last test finished" depending on the environment.
I wan to propose an alternative, serialize tests so that timeouts are predictable to "the prior test finished". I'm not 100% sure what the best implementation for this is. Or, perhaps even better, the timeout defines how long the test is allowed to take instead of how long the prior test is allowed to take (question here: is that overlapping with what cargo test already does?)
I think it would require somewhat of implementation of a custom test runner that collects all tests for a given key and runs them sequentially.
Related to earlier question in https://github.com/palfrey/serial_test/issues/68
https://github.com/palfrey/serial_test/pull/83 resolves this by removing timeouts entirely