Test timeouts

[clarfonthey]

It would be really nice if Rust had some kind of built-in timeout functionality, e.g.:

panic_after(Duration::from_millis(100), || {
    do_a_thing()
}

The API could be something different, maybe, and maybe there'd be a way to integrate it with the thread API instead of the test API. But either way, it would be nice to have to import an external crate to have some sort of maximum runtime for tests, to avoid infinite loops destroying the test runtime.

[shepmaster]

I'd expect anything here to mostly happen in custom test frameworks. There's also the problem that you can't kill a thread, so anything either has to be cooperative or allow zombie threads.

[Lokathor]

I know you can terminate a thread on windows. Do linux and mac lack that ability?

[MikailBag]

Linux supports it via tgkill(2) (AFAIK ok, but no cleanup) or pthread_cancel(3) (as far as I heard on rust-lang zulip, pthread_cancel can trigger Rust UB).

[clarfonthey]

Honestly, zombie threads are better than living threads in an infinite loop. They can be reaped when the process exits in the case of tests, at least.

[canndrew]

I'd expect anything here to mostly happen in custom test frameworks.

The default test framework that ships with rust should support this though. It's a small but necessary feature.

I think the timeout should be passed to the #[test] attribute as an argument. eg.

#[test(timeout = "200ms")]
fn my_cool_test() {}

or

#[test(timeout = Duration::new(1, 0))]
fn my_cool_test() {}

[shepmaster]

It's a small but necessary feature

I'll argue about necessary as we've lived without it for 4+ years and it can be written as a helper method today:

use std::{sync::mpsc, thread, time::Duration};

#[test]
fn oops() {
    panic_after(Duration::from_millis(100), || {
        thread::sleep(Duration::from_millis(200));
    })
}

fn panic_after<T, F>(d: Duration, f: F) -> T
where
    T: Send + 'static,
    F: FnOnce() -> T,
    F: Send + 'static,
{
    let (done_tx, done_rx) = mpsc::channel();
    let handle = thread::spawn(move || {
        let val = f();
        done_tx.send(()).expect("Unable to send completion signal");
        val
    });

    match done_rx.recv_timeout(d) {
        Ok(_) => handle.join().expect("Thread panicked"),
        Err(_) => panic!("Thread took too long"),
    }
}

Nice to have? Certainly.

[sasa-tomic]

I'll argue about necessary as we've lived without it for 4+ years and it can be written as a helper method today

While it's possible to have the test timeout reduced this way, it's not possible to increase the test timeout over the hard-coded 120s value.

For our project (and surely for some other projects as well), we have some long-running tests for which 120s in some cases isn't enough. We need something like

#[test(timeout = Duration::new(300, 0))]
fn my_cool_test() {}

And this can't be done with the helper method IIUC.

[Yoga07]

Just in case somebody is interested in this: ntest crate has the #[timeout(_)] attr that can timeout tests and panic accordingly.

[rukai]

While it's possible to have the test timeout reduced this way, it's not possible to increase the test timeout over the hard-coded 120s value.

This is the first result on google for "rust test timeout" so I need to point out that there is no hard-coded 120s timeout. The linked code is a nightly only feature and it must be enabled via cargo test -- -Z --ensure-time. I even verified that tests will still pass after sleeping for 400s

[Pr0methean]

One option besides zombie threads would be to split the process into two: a worker that executes the test code, and a supervisor that gathers the results and kills timed-out workers. That's what JUnit does.

[mathstuf]

Sounds like something in the vein of nextest then, no?