test-case icon indicating copy to clipboard operation
test-case copied to clipboard

Declare test cases once, use them multiple times

Open thatbakamono opened this issue 2 years ago • 4 comments

Is there a way to declare test cases only once and use them multiple times across multiple functions in multiple files? I read the wiki but I didn't see anything like that. If there isn't, that might be a feature worth adding.

thatbakamono avatar Apr 03 '22 15:04 thatbakamono

Sorry for late reply - busy times. How would this look like? Could you provide examples / usages in other test frameworks?

luke-biel avatar Apr 09 '22 22:04 luke-biel

I'm not sure what is the best way to realise that, some ways are harder but a lot more configurable and some are simpler but less flexible, It's up to debate which one is the best for test-case.

rstest does that in a really simple but at the same time not so powerful way:

You first declare a template as an empty test function marked using #[template] with test cases you want to use multiple times

#[template]
#[rstest]
#[case(2, 2)]
#[case(4/2, 2)]
fn two_simple_cases(#[case] a: u32, #[case] b: u32) {}

and then you can apply it to a bunch of other tests using #[apply]

#[apply(two_simple_cases)]
fn it_works(#[case] a: u32, #[case] b: u32) {
    assert!(a == b);
}

proptest's way is a bit less simple but it's also a lot more powerful:

You first declare a strategy (basically a function generating values):

fn vec_and_index() -> impl Strategy<Value = (Vec<String>, usize)> {
    prop::collection::vec(".*", 1..100)
        .prop_flat_map(|vec| {
            let len = vec.len();
            (Just(vec), 0..len)
        })
}

and then you can use it as an input in your tests

proptest! {
    #[test]
    fn test_some_function((vec, index) in vec_and_index()) {
        some_function(vec, index);
    }
}

fn some_function(stuff: Vec<String>, index: usize) {
    let _ = &stuff[index];
    // Do stuff
}

thatbakamono avatar Apr 10 '22 19:04 thatbakamono

Okay, now I have a better picture.
I'm more fond of second design especially that a value generator is something that has gone through my mind before. A blocker in this case is how we structured our code - it's a singular proc-macro crate. I'm trying to rewrite it so that we could export other items than just macros, but it's gonna take some time.
After that we can definitelly design this functionality in terms of test-case and add it.

luke-biel avatar Apr 11 '22 19:04 luke-biel

I've just found this issue after asking about this on the rust forum.

Is there a concrete reason why this could not be done with compositional macros. I'm new to rust so I'm most likely aware of something specific in the way macros work.

Ideally something along these lines would make the existing framework much more powerful.

   macro_rules! foo {
        () => {
            [
                (0, &[0x00]),
                (127, &[0x7F]),
                (128, &[0x80, 0x01])
            ]
        };
    }

    #[test_matrix( foo!() )]
    fn bar(example: (u32, &[u8])) {
        // ...
    }

Due to reasons that I don't understand, this expands in such a way as to try to invoke bar with the entire array, rather than interpret the output of foo as a token stream. Is this form entirely impossible in rust?

couling avatar Aug 13 '24 13:08 couling