Extract e2e testing framework to its own crate

[ascjones]

A WIP experiment for now just to see whether it is possible. If it is it will resolve #1428.

Pulls all of the e2e testing framework and the macro into their own crates.

• Keeps all e2e related code and dependencies together
• Only builds e2e dependencies for testing, prevents pulling in a big dependency tree for ink crates

[cmichi]

Thanks a lot for this! I don't see how it will resolve https://github.com/paritytech/ink/issues/1428, but we should merge it nonetheless, as it's more structured this way.

I don't think it will resolve our issue because of the following:

• E.g. flipper includes the ink dependency.
• You execute cargo test on flipper.
• This will result in the direct dev-dependencies defined in flipper/Cargo.toml being built.
• However: The ink crate will be built without the information that flipper is built in a developer/testing environment. So you have no way to know in the ink crate if it is being built for dev-dependencies or regular dependencies.
• This is unfortunately documented behavior (here):

Dev-dependencies are not used when compiling a package for building, but are used for compiling tests, examples, and benchmark. These dependencies are not propagated to other packages which depend on this package.

I think what we have to do is require every contract to have a Cargo.toml like this:

[dependencies]
ink = { version = "4.0", default-features = false }

[dev-dependencies]
ink = { version = "4.0", default-features = false, features = ["as-dev-dependency"] }

The we can have the crates/ink/Cargo.toml contain:

[features]
as-dev-dependency = [
   "ink_e2e"
]

[ascjones]

I don't see how it will resolve https://github.com/paritytech/ink/issues/1428

The idea is that ink_e2e is a standalone crate (not reexported by the main ink crate).

[dev-dependencies]
ink_e2e = { path = "../../crates/e2e" }

[ascjones]

Waiting for https://github.com/paritytech/cargo-contract/pull/763 to be merged and released, then update ink! CI in this PR

[codecov-commenter]

Codecov Report

Merging #1429 (791afbc) into master (a4594f0) will increase coverage by 0.05%. The diff coverage is 7.14%.

@@            Coverage Diff             @@
##           master    #1429      +/-   ##
==========================================
+ Coverage   70.84%   70.89%   +0.05%     
==========================================
  Files         193      193              
  Lines        6077     6071       -6     
==========================================
- Hits         4305     4304       -1     
+ Misses       1772     1767       -5     

Impacted Files	Coverage Δ
crates/e2e/macro/src/codegen.rs	0.00% <0.00%> (ø)
crates/e2e/macro/src/config.rs	45.65% <ø> (ø)
crates/e2e/macro/src/ir.rs	0.00% <0.00%> (ø)
crates/e2e/macro/src/lib.rs	0.00% <0.00%> (ø)
crates/e2e/src/lib.rs	0.00% <ø> (ø)
crates/e2e/src/xts.rs	0.00% <ø> (ø)
crates/ink/codegen/src/generator/mod.rs	100.00% <ø> (ø)
crates/ink/codegen/src/lib.rs	100.00% <ø> (ø)
crates/ink/macro/src/lib.rs	100.00% <ø> (+11.11%)	:arrow_up:
crates/ink/ir/src/ir/utils.rs	92.59% <100.00%> (ø)
... and 1 more

:mega: We’re building smart automated test selection to slash your CI/CD build times. Learn more