bolero icon indicating copy to clipboard operation
bolero copied to clipboard

macros: add test macro

Open camshaft opened this issue 3 years ago • 4 comments

closes #39

camshaft avatar Nov 09 '21 09:11 camshaft

Just thinking: would it make sense, in addition to this, to have a #[bolero::non_fuzz_test] or similar macro?

This’d allow the tests to be skipped when run with BOLERO_LIBFUZZER_ARGS, CARGO_BOLERO_SELECT, or any similar environment variable, and thus to make listing probably significantly faster

Ekleog-NEAR avatar Jul 12 '23 12:07 Ekleog-NEAR

I'd really like to fix the listing in a different way, ideally by embedding sections in the output artifact and cargo-bolero reading from those. This way we don't have to run the program at all and the user doesn't have to go annotate a bunch of slower tests.

camshaft avatar Jul 12 '23 14:07 camshaft

That’d definitely be a much better solution, you’re right!

Ekleog-NEAR avatar Jul 12 '23 14:07 Ekleog-NEAR

I thought about it some more, and it turns out that:

  • Linker-based test listing is hard
  • If a #[bolero::test] macro is added, we no longer need bolero::check!()
  • With a #[bolero::test] proc macro, making a list becomes trivial: just have the proc macro output an additional item #[test] fn __bolero_internal_list__{test_name}() that just outputs the JSON information, and run cargo test __bolero_internal_list__

As such, I think a reasonable way forward would be:

  • Add a #[bolero::test] macro that in addition to the test, also adds the __bolero_internal_list__{test_name}
  • Remove the current bolero::check macro
  • Make cargo list use cargo test __bolero_internal_list__

WDYT?

Ekleog-NEAR avatar Jul 14 '23 09:07 Ekleog-NEAR