Fix timeout not being enforced by sync_until

[conradoplg]

Motivation

The sync_until function, which is used by e.g. sync_one_checkpoint_testnet, does not enforce timeouts.

This makes hangs in the underlying issue harder to diagnose, since CI will only fail after the 6 hour timeout.

Make it always enforce the timeout.

Specifications

Designs

This is caused by wait_with_output being called when mempool activation is not required, and that function does not enforce timeouts (as documented).

Stop using wait_with_output:

• [ ] Change sync_until to check the mempool didn't activate using with_failure_regex_iter()
• [ ] Remove wait_with_output from sync_until, most of the checks are the same, only the mempool checks are different
• [ ] After it's fixed, if possible, ensure that if the test fails, the captured output is printed to the terminal. This will make debugging issues much easier.

Related Work

[teor2345]

Find out if wait_with_output needs to not enforce timeout, or if we just didn't add support at the time. Depending on the answer, make it enforce timeouts (one way to do that), or change sync_until to use another approach.

It's not possible to enforce timeouts using std::TestChild.

I've edited the ticket description to use with_failure_regex_iter - we didn't have that test harness feature when we added mempool support to sync_until.

We could also use process_control as a general fix for timeouts, and to simplify our code. But that seems like a bigger ticket.

[conradoplg]

I've edited the ticket description to use with_failure_regex_iter - we didn't have that test harness feature when we added mempool support to sync_until.

Thanks!

[teor2345]

This test bug does not block any of our release work.

[teor2345]

This isn't actually causing us any problems at the moment