implementing cabbage tests that use in_beam_watcher and mix_based_child_chain

[InoMurko]

elixir-omg  inomurko/ife2 $ grep -r ":mix_based_child_chain" apps/
apps//omg_performance/test/omg_performance/byzantine_event_test.exs:  @tag fixtures: [:perf_test, :mix_based_child_chain, :mix_based_watcher]
apps//omg_performance/test/omg_performance/byzantine_event_test.exs:  @tag fixtures: [:perf_test, :mix_based_child_chain, :mix_based_watcher]
apps//omg_performance/test/omg_performance/byzantine_event_test.exs:  @tag fixtures: [:perf_test, :mix_based_child_chain, :mix_based_watcher]
apps//omg_performance/test/omg_performance/byzantine_event_test.exs:  @tag fixtures: [:perf_test, :mix_based_child_chain, :mix_based_watcher]
apps//omg_performance/test/omg_performance/byzantine_event_test.exs:  @tag fixtures: [:perf_test, :mix_based_child_chain, :mix_based_watcher]
apps//omg_watcher/test/omg_watcher/integration/transaction_submit_test.exs:  @tag fixtures: [:in_beam_watcher, :mix_based_child_chain, :stable_alice, :bob, :stable_alice_deposits]
apps//omg_watcher/test/omg_watcher/integration/invalid_exit_test.exs:  @tag fixtures: [:in_beam_watcher, :stable_alice, :mix_based_child_chain, :token, :stable_alice_deposits]
apps//omg_watcher/test/omg_watcher/integration/invalid_exit_test.exs:  @tag fixtures: [:in_beam_watcher, :stable_alice, :mix_based_child_chain, :token, :stable_alice_deposits, :test_server]
apps//omg_watcher/test/omg_watcher/integration/invalid_exit_test.exs:  @tag fixtures: [:in_beam_watcher, :stable_alice, :mix_based_child_chain, :token, :stable_alice_deposits]
apps//omg_watcher/test/omg_watcher/integration/in_flight_exit_test.exs:  @tag fixtures: [:in_beam_watcher, :alice, :bob, :mix_based_child_chain, :token, :alice_deposits]
apps//omg_watcher/test/omg_watcher/integration/in_flight_exit_test.exs:  @tag fixtures: [:in_beam_watcher, :alice, :bob, :mix_based_child_chain, :token, :alice_deposits]
apps//omg_watcher/test/omg_watcher/integration/in_flight_exit_test.exs:  @tag fixtures: [:in_beam_watcher, :alice, :bob, :mix_based_child_chain, :token, :alice_deposits]
apps//omg_watcher/test/omg_watcher/integration/in_flight_exit_test.exs:  @tag fixtures: [:in_beam_watcher, :alice, :bob, :mix_based_child_chain, :token, :alice_deposits]
apps//omg_watcher/test/omg_watcher/integration/in_flight_exit_test.exs:  @tag fixtures: [:in_beam_watcher, :alice, :bob, :mix_based_child_chain, :token, :alice_deposits]
apps//omg_watcher/test/omg_watcher/integration/block_getter_test.exs:  @tag fixtures: [:in_beam_watcher, :mix_based_child_chain, :alice, :bob, :alice_deposits, :token]
apps//omg_watcher/test/omg_watcher/integration/block_getter_test.exs:  @tag fixtures: [:in_beam_watcher, :stable_alice, :mix_based_child_chain, :token, :stable_alice_deposits, :test_server]
elixir-omg  inomurko/ife2 $
elixir-omg  inomurko/ife2 $
elixir-omg  inomurko/ife2 $ grep -r ":mix_based_watcher" apps/
apps//omg_performance/test/omg_performance/byzantine_event_test.exs:  @tag fixtures: [:perf_test, :mix_based_child_chain, :mix_based_watcher]
apps//omg_performance/test/omg_performance/byzantine_event_test.exs:  @tag fixtures: [:perf_test, :mix_based_child_chain, :mix_based_watcher]
apps//omg_performance/test/omg_performance/byzantine_event_test.exs:  @tag fixtures: [:perf_test, :mix_based_child_chain, :mix_based_watcher]
apps//omg_performance/test/omg_performance/byzantine_event_test.exs:  @tag fixtures: [:perf_test, :mix_based_child_chain, :mix_based_watcher]
apps//omg_performance/test/omg_performance/byzantine_event_test.exs:  @tag fixtures: [:perf_test, :mix_based_child_chain, :mix_based_watcher]
elixir-omg  inomurko/ife2 $ grep -r ":in_beam_watcher" apps/
apps//omg_watcher/test/omg_watcher/integration/transaction_submit_test.exs:  @tag fixtures: [:in_beam_watcher, :mix_based_child_chain, :stable_alice, :bob, :stable_alice_deposits]
apps//omg_watcher/test/omg_watcher/integration/invalid_exit_test.exs:  @tag fixtures: [:in_beam_watcher, :stable_alice, :mix_based_child_chain, :token, :stable_alice_deposits]
apps//omg_watcher/test/omg_watcher/integration/invalid_exit_test.exs:  @tag fixtures: [:in_beam_watcher, :stable_alice, :mix_based_child_chain, :token, :stable_alice_deposits, :test_server]
apps//omg_watcher/test/omg_watcher/integration/invalid_exit_test.exs:  @tag fixtures: [:in_beam_watcher, :stable_alice, :mix_based_child_chain, :token, :stable_alice_deposits]
apps//omg_watcher/test/omg_watcher/integration/status_test.exs:  @tag fixtures: [:in_beam_watcher, :root_chain_contract_config]
apps//omg_watcher/test/omg_watcher/integration/in_flight_exit_test.exs:  @tag fixtures: [:in_beam_watcher, :alice, :bob, :mix_based_child_chain, :token, :alice_deposits]
apps//omg_watcher/test/omg_watcher/integration/in_flight_exit_test.exs:  @tag fixtures: [:in_beam_watcher, :alice, :bob, :mix_based_child_chain, :token, :alice_deposits]
apps//omg_watcher/test/omg_watcher/integration/in_flight_exit_test.exs:  @tag fixtures: [:in_beam_watcher, :alice, :bob, :mix_based_child_chain, :token, :alice_deposits]
apps//omg_watcher/test/omg_watcher/integration/in_flight_exit_test.exs:  @tag fixtures: [:in_beam_watcher, :alice, :bob, :mix_based_child_chain, :token, :alice_deposits]
apps//omg_watcher/test/omg_watcher/integration/in_flight_exit_test.exs:  @tag fixtures: [:in_beam_watcher, :alice, :bob, :mix_based_child_chain, :token, :alice_deposits]
apps//omg_watcher/test/omg_watcher/integration/block_getter_test.exs:  @tag fixtures: [:in_beam_watcher, :mix_based_child_chain, :alice, :bob, :alice_deposits, :token]
apps//omg_watcher/test/omg_watcher/integration/block_getter_test.exs:  @tag fixtures: [:in_beam_watcher, :test_server]
apps//omg_watcher/test/omg_watcher/integration/block_getter_test.exs:  @tag fixtures: [:in_beam_watcher, :alice, :test_server]
apps//omg_watcher/test/omg_watcher/integration/block_getter_test.exs:  @tag fixtures: [:in_beam_watcher, :stable_alice, :mix_based_child_chain, :token, :stable_alice_deposits, :test_server]

Some of the tests above re-occur as they use both in_beam_watcher and mix_based_child_chain

[InoMurko]

This will probably need to solve another problem: Shared state between tests. The tests above run in a way where the setup of the test is basically a clean slate.

[pdobacz]

apps//omg_watcher/test/omg_watcher/integration/invalid_exit_test.exs:

some got added to cabbage in #1344.

Shared state between tests. The tests above run in a way where the setup of the test is basically a clean slate.

:+1: A follow-up comment: When doing invalid_exit_test.exs in cabbage in a recent PR https://github.com/omisego/elixir-omg/pull/1344 I stumbled upon this very problem.

Tentatively, I fixed this by making sure that those tests added there (to cabbage, that is) as well as the pre-existing in_flight_exits_test.exs cabbage test left the chain in a "clean" state, at least as far as byzantine_events go.

In in_flight_exits_test.exs this meant for instance piggybacking all inputs/outputs of IFE and challenging them. For invalid_standard_exits_test.exs this meant challenging :unchallenged_exits.

However, this still falls short, when speaking about conditions that invalidate the chain permanently (invalid block, invalidly processed exit (see #1318)).

[InoMurko]

@pdobacz question: the #1344 added some test both in cabbage and in elixir-omg. are we able to delete any tests that got implemented as cabbage tests now?

[pdobacz]

@InoMurko

@pdobacz question: the #1344 added some test both in cabbage and in elixir-omg. are we able to delete any tests that got implemented as cabbage tests now?

It didn't really add non-cabbage tests (as in, add new behaviors/features tested; it just tidied them) and I postponed removing non-cabbage tests.

As per PR description there:

• does some necessary intro fixes to the "old" integration tests which were not strict enough
• separates those tests into two "kinds"
  • tests requiring an invalid block (not moving to cabbage yet)
  • tests not requiring an invalid block (this is moving)
• turns three of those tests into cabbage tests

and

I'm keeping the tidied up "old" approach tests, as I'd like to first gain more confidence in the cabbage tests added.

[InoMurko]

ahaaa, okay, I only took a quick peek at the tests. are we able to identify and mark potential tests that are implemented in both "old" way and cabbage? just so that we know what tests have coverage in both implementations... I want to avoid guessing what we can delete and what not.

[pdobacz]

ahaaa, okay, I only took a quick peek at the tests. are we able to identify and mark potential tests that are implemented in both "old" way and cabbage?

TBH, I think the "old" way and cabbage diverge far enough, in terms of how they test, that there is no 1:1 relationship between them.

For now, I used this issue to comment on work done in #1344 related to the "old" invalid_exit_test.exs :point_up: .

But I think, in the long run, this issue is going to be done more or less like this:

• top-down design thinking on how cabbage is going to be structured/phrased (in gherkin)/split/setup-torndown
• write comprehensive cabbage tests aligning to the design
• carefully go through the "old" way tests and figure out, whether removing the "old" way tests will not leave gaps in coverage (possibly tweak cabbage in this process)

So guessing will be avoided.