ink icon indicating copy to clipboard operation
ink copied to clipboard

Add ability to use quickcheck with E2E testing

Open cmichi opened this issue 2 years ago • 1 comments

We should allow for the possibility of using quickcheck with our E2E testing framework.

The problem here is that the #[quickcheck] macro already generates code which contains the #[test] attribute (whereas we need #[ink_e2e::test]. But I think we can access the quickcheck functionality via some other way directly calling functions on the crate.

ToDo 1

Please investigate if we can find a way to make this work:

#[quickcheck]
#[ink_e2e::test]

I think having this :point_up: would be the best API, but I think it's unlikely that we can get this to work. The other way would be sth like

#[ink_e2e::test(quickcheck = …)]

and then pass the arguments through. So quickcheck would be a direct dependency of ink_e2e and we would likely call a Rust function directly on that crate.

ToDo 2: Benchmark

After https://github.com/paritytech/ink/pull/1642 I'm not sure how running quickcheck on top of our E2E testing will behave. We should have some numbers of how long it takes to e.g. fuzz a contract message 10_000 times.

cmichi avatar Feb 09 '23 20:02 cmichi

@cmichi After some digging around: we can use quickcheck(<func_name> as <func_signature>) directly in our E2E tests as shown here. It behaves exactly as macros. However, that would require "diving" into the test function block and parsing build_message calls

e.g.

let insert = ink_e2e::build_message::<MappingsRef>(contract_id)
    .call(|contract| contract.insert_balance(1_000));
let size = client
    .call(&ink_e2e::alice(), insert, 0, None)
    .await
    .expect("Calling `insert_balance` failed")
    .return_value();

and turning them into sub-functions which can be accepted as args by quickcheck. That is actually where #1644 would come handy as it would simplify parsing because contract's associated functions will have a simple mapping of primitive types (e.g. usize -> usize), and we can plug them directly into quickcheck function

Otherwise, I personally can't see another way of doing it or using quickcheck! macro or #[quickcheck] attr without changing the API of E2E

SkymanOne avatar Feb 10 '23 03:02 SkymanOne