Broken coverage testing in af-cli

Open alnoki opened this issue 2 years ago • 0 comments

Overview

As described in #1275, the aptos CLI does not support coverage testing, and since the move CLI does not support AptosFramework native functions, coverage testing is thus prohibited for code that is not implemented purely in Move.

A potential workaround is the af-cli, which purportedly mimics the behavior of the move CLI with added support for AptosFramework native functions, but the tool is broken:

Reproducible behavior

Using git commit 684144f2c4654c62d5bac202cfa014fd00b9f7bf for https://github.com/econia-labs/econia, from inside src/move/econia, with environment variable RUST_BACKTRACE=full:

Test execution failure

(econia) econia % af-cli package test --coverage
INCLUDING DEPENDENCY AptosFramework
INCLUDING DEPENDENCY MoveStdlib
BUILDING Econia
Running Move unit tests
[ PASS    ] 0xc0deb00c::Book::add_position_success_crossed_spread_ask
[ PASS    ] 0xc0deb00c::CritBit::b_lo_success
[ PASS    ] 0xc0deb00c::Caps::book_f_c_failure
[ PASS    ] 0xc0deb00c::CritBit::borrow_empty
[ PASS    ] 0xc0deb00c::Version::get_v_n_success
thread '<unnamed>' panicked at 'extension unknown', /Users/user/.cargo/git/checkouts/move-0639dd674f581c30/ece13ae/language/move-vm/runtime/src/native_extensions.rs:32:14
stack backtrace:
thread '<unnamed>[ PASS    ] 0xc0deb00c::Init::init_econia_failure_not_econia
' panicked at 'extension unknown', /Users/user/.cargo/git/checkouts/move-0639dd674f581c30/ece13ae/language/move-vm/runtime/src/native_extensions.rs:32:14
[ PASS    ] 0xc0deb00c::CritBit::borrow_mut_empty
   0:        0x10633d322 - <std::sys_common::backtrace::_print::DisplayBacktrace as core::fmt::Display>::fmt::h16d1a5d48aeff79a
   1:        0x106356a1a - core::fmt::write::hd4894418914ead0b
   2:        0x106325808 - std::io::Write::write_fmt::h64d317bb018e2b5c
   3:        0x10632ae1d - std::panicking::default_hook::{{closure}}::h4fa7fb9660e915bb
   4:        0x10632aad5 - std::panicking::default_hook::hfc7c802f5856fff7
   5:        0x10632b49f - std::panicking::rust_panic_with_hook::hac34d5423636aa62
   6:        0x10633da73 - std::panicking::begin_panic_handler::{{closure}}::h20649726a6fa6320
   7:        0x10633d437 - std::sys_common::backtrace::__rust_end_short_backtrace::hc440a536609cc112
   8:        0x10632af63 - _rust_begin_unwind
   9:        0x1063d8533 - core::panicking::panic_fmt::h72d82fb70320b112
  10:        0x10636005b - core::panicking::panic_display::h6f63c756fdbb722c
  11:        0x10636000c - core::panicking::panic_str::hae0c49c3a431d341
  12:        0x1063d8629 - core::option::expect_failed::h7ad4202aa9d6d124
  13:        0x1052c992e - move_vm_runtime::native_extensions::NativeContextExtensions::get::hfb5c6fbd90ce6070
  14:        0x1052c580b - move_table_extension::native_new_table_handle::hb08084c5b24ccce6
  15:        0x1059c67bd - move_vm_runtime::interpreter::Interpreter::call_native::h17396793e89b4f0d
  16:        0x1054e8723 - move_vm_runtime::interpreter::Interpreter::entrypoint::h9762d10306c8c846
  17:        0x1054c090c - move_vm_runtime::runtime::VMRuntime::execute_function::hda8604d783290956
  18:        0x1054fff98 - move_unit_test::test_runner::SharedTestingConfig::execute_via_move_vm::h4f1a0e28b150a1f3
  19:        0x1053f9df7 - core::ops::function::impls::<impl core::ops::function::FnMut<A> for &F>::call_mut::h642aed78e2a57976
  20:        0x105383c10 - <core::iter::adapters::map::Map<I,F> as core::iter::traits::iterator::Iterator>::fold::h63f54264f81342ef
  21:        0x10534c97a - rayon::iter::plumbing::bridge_producer_consumer::helper::hd8f9a7865a28cb0c
  22:        0x10533fdcf - rayon_core::join::join_context::{{closure}}::h541557b102a3c8a9
  23:        0x10534cb07 - rayon::iter::plumbing::bridge_producer_consumer::helper::hd8f9a7865a28cb0c
  24:        0x10543df5f - <rayon_core::job::StackJob<L,F,R> as rayon_core::job::Job>::execute::h82825343aa506a34
  25:        0x1063893d7 - rayon_core::registry::WorkerThread::wait_until_cold::hffe75213be8f2c8b
  26:        0x105746119 - rayon_core::registry::ThreadBuilder::run::h1b1a2a03b1f3e2a1
  27:        0x1057490f2 - std::sys_common::backtrace::__rust_begin_short_backtrace::hc20dbe2184480a24
  28:        0x1057431f5 - core::ops::function::FnOnce::call_once{{vtable.shim}}::h6b930a553643aea7
  29:        0x106330237 - std::sys::unix::thread::Thread::new::thread_start::he611271e49184318
  30:     0x7ff81afb64e1 - __pthread_start
stack backtrace:
   0:        0x10633d322 - <std::sys_common::backtrace::_print::DisplayBacktrace as core::fmt::Display>::fmt::h16d1a5d48aeff79a
   1:        0x106356a1a - core::fmt::write::hd4894418914ead0b
   2:        0x106325808 - std::io::Write::write_fmt::h64d317bb018e2b5c
   3:        0x10632ae1d - std::panicking::default_hook::{{closure}}::h4fa7fb9660e915bb
   4:        0x10632aad5 - std::panicking::default_hook::hfc7c802f5856fff7
   5:        0x10632b49f - std::panicking::rust_panic_with_hook::hac34d5423636aa62
   6:        0x10633da73 - std::panicking::begin_panic_handler::{{closure}}::h20649726a6fa6320
   7:        0x10633d437 - std::sys_common::backtrace::__rust_end_short_backtrace::hc440a536609cc112
   8:        0x10632af63 - _rust_begin_unwind
   9:        0x1063d8533 - core::panicking::panic_fmt::h72d82fb70320b112
  10:        0x10636005b - core::panicking::panic_display::h6f63c756fdbb722c
  11:        0x10636000c - core::panicking::panic_str::hae0c49c3a431d341
  12:        0x1063d8629 - core::option::expect_failed::h7ad4202aa9d6d124
  13:        0x1052c992e - move_vm_runtime::native_extensions::NativeContextExtensions::get::hfb5c6fbd90ce6070
  14:        0x1052c580b - move_table_extension::native_new_table_handle::hb08084c5b24ccce6
  15:        0x1059c67bd - move_vm_runtime::interpreter::Interpreter::call_native::h17396793e89b4f0d
  16:        0x1054e8723 - move_vm_runtime::interpreter::Interpreter::entrypoint::h9762d10306c8c846
  17:        0x1054c090c - move_vm_runtime::runtime::VMRuntime::execute_function::hda8604d783290956
  18:        0x1054fff98 - move_unit_test::test_runner::SharedTestingConfig::execute_via_move_vm::h4f1a0e28b150a1f3
  19:        0x1053f9df7 - core::ops::function::impls::<impl core::ops::function::FnMut<A> for &F>::call_mut::h642aed78e2a57976
  20:        0x105383c10 - <core::iter::adapters::map::Map<I,F> as core::iter::traits::iterator::Iterator>::fold::h63f54264f81342ef
  21:        0x10534c97a - rayon::iter::plumbing::bridge_producer_consumer::helper::hd8f9a7865a28cb0c
  22:        0x10533fdcf - rayon_core::join::join_context::{{closure}}::h541557b102a3c8a9
  23:        0x10534cb07 - rayon::iter::plumbing::bridge_producer_consumer::helper::hd8f9a7865a28cb0c
  24:        0x10543df5f - <rayon_core::job::StackJob<L,F,R> as rayon_core::job::Job>::execute::h82825343aa506a34
  25:        0x1063893d7 - rayon_core::registry::WorkerThread::wait_until_cold::hffe75213be8f2c8b
  26:        0x105746119 - rayon_core::registry::ThreadBuilder::run::h1b1a2a03b1f3e2a1
  27:        0x1057490f2 - std::sys_common::backtrace::__rust_begin_short_backtrace::hc20dbe2184480a24
  28:        0x1057431f5 - core::ops::function::FnOnce::call_once{{vtable.shim}}::h6b930a553643aea7
  29:        0x106330237 - std::sys::unix::thread::Thread::new::thread_start::he611271e49184318
  30:     0x7ff81afb64e1 - __pthread_start
[ PASS    ] 0xc0deb00c::CritBit::borrow_mut_no_match
[ PASS    ] 0xc0deb00c::Orders::add_order_failure_no_orders
[ PASS    ] 0xc0deb00c::Book::add_position_success_crossed_spread_bid
thread '<unnamed>' panicked at 'extension unknown', /Users/user/.cargo/git/checkouts/move-0639dd674f581c30/ece13ae/language/move-vm/runtime/src/native_extensions.rs:32:14
stack backtrace:
   0:        0x10633d322 - <std::sys_common::backtrace::_print::DisplayBacktrace as core::fmt::Display>::fmt::h16d1a5d48aeff79a
   1:        0x106356a1a - core::fmt::write::hd4894418914ead0b
   2:        0x106325808 - std::io::Write::write_fmt::h64d317bb018e2b5c
   3:        0x10632ae1d - std::panicking::default_hook::{{closure}}::h4fa7fb9660e915bb
   4:        0x10632aad5 - std::panicking::default_hook::hfc7c802f5856fff7
   5:        0x10632b49f - std::panicking::rust_panic_with_hook::hac34d5423636aa62
   6:        0x10633da73 - std::panicking::begin_panic_handler::{{closure}}::h20649726a6fa6320
   7:        0x10633d437 - std::sys_common::backtrace::__rust_end_short_backtrace::hc440a536609cc112
   8:        0x10632af63 - _rust_begin_unwind
   9:        0x1063d8533 - core::panicking::panic_fmt::h72d82fb70320b112
  10:        0x10636005b - core::panicking::panic_display::h6f63c756fdbb722c
  11:        0x10636000c - core::panicking::panic_str::hae0c49c3a431d341
  12:        0x1063d8629 - core::option::expect_failed::h7ad4202aa9d6d124
  13:        0x1052c992e - move_vm_runtime::native_extensions::NativeContextExtensions::get::hfb5c6fbd90ce6070
  14:        0x1052c580b - move_table_extension::native_new_table_handle::hb08084c5b24ccce6
  15:        0x1059c67bd - move_vm_runtime::interpreter::Interpreter::call_native::h17396793e89b4f0d
  16:        0x1054e8723 - move_vm_runtime::interpreter::Interpreter::entrypoint::h9762d10306c8c846
  17:        0x1054c090c - move_vm_runtime::runtime::VMRuntime::execute_function::hda8604d783290956
  18:        0x1054fff98 - move_unit_test::test_runner::SharedTestingConfig::execute_via_move_vm::h4f1a0e28b150a1f3
  19:        0x1053f9df7 - core::ops::function::impls::<impl core::ops::function::FnMut<A> for &F>::call_mut::h642aed78e2a57976
  20:        0x105383c10 - <core::iter::adapters::map::Map<I,F> as core::iter::traits::iterator::Iterator>::fold::h63f54264f81342ef
  21:        0x10534c97a - rayon::iter::plumbing::bridge_producer_consumer::helper::hd8f9a7865a28cb0c
  22:        0x10533fdcf - rayon_core::join::join_context::{{closure}}::h541557b102a3c8a9
  23:        0x10534cb07 - rayon::iter::plumbing::bridge_producer_consumer::helper::hd8f9a7865a28cb0c
  24:        0x10533fdcf - rayon_core::join::join_context::{{closure}}::h541557b102a3c8a9
  25:        0x10534cb07 - rayon::iter::plumbing::bridge_producer_consumer::helper::hd8f9a7865a28cb0c
  26:        0x10543df5f - <rayon_core::job::StackJob<L,F,R> as rayon_core::job::Job>::execute::h82825343aa506a34
  27:        0x1063893d7 - rayon_core::registry::WorkerThread::wait_until_cold::hffe75213be8f2c8b
  28:        0x105746119 - rayon_core::registry::ThreadBuilder::run::h1b1a2a03b1f3e2a1
  29:        0x1057490f2 - std::sys_common::backtrace::__rust_begin_short_backtrace::hc20dbe2184480a24
  30:        0x1057431f5 - core::ops::function::FnOnce::call_once{{vtable.shim}}::h6b930a553643aea7
  31:        0x106330237 - std::sys::unix::thread::Thread::new::thread_start::he611271e49184318
  32:     0x7ff81afb64e1 - __pthread_start
[ PASS    ] 0xc0deb00c::Orders::add_order_failure_not_multiple
[ PASS    ] 0xc0deb00c::Orders::add_order_failure_overflow
[ PASS    ] 0xc0deb00c::Version::init_mock_failure_exists
[ PASS    ] 0xc0deb00c::Orders::add_order_failure_price_0
[ PASS    ] 0xc0deb00c::Orders::add_order_failure_size_0
[ PASS    ] 0xc0deb00c::Version::init_mock_failure_not_econia
[ PASS    ] 0xc0deb00c::Caps::has_f_c_success
[ PASS    ] 0xc0deb00c::Orders::add_orders_success
[ PASS    ] 0xc0deb00c::Book::add_position_success_simple_ask
[ PASS    ] 0xc0deb00c::Caps::init_caps_failure_exists
[ PASS    ] 0xc0deb00c::Orders::get_friend_cap_failure
[ PASS    ] 0xc0deb00c::Version::init_mock_success
[ PASS    ] 0xc0deb00c::Orders::get_friend_cap_success
[ PASS    ] 0xc0deb00c::CritBit::borrow_mut_success
[ PASS    ] 0xc0deb00c::Book::add_position_success_simple_bid
[ PASS    ] 0xc0deb00c::Orders::init_orders_failure_exists
[ PASS    ] 0xc0deb00c::Caps::init_caps_failure_not_econia
[ PASS    ] 0xc0deb00c::Book::get_friend_cap_failure
[ PASS    ] 0xc0deb00c::CritBit::borrow_no_match
[ PASS    ] 0xc0deb00c::Orders::init_orders_success
[ PASS    ] 0xc0deb00c::CritBit::check_len_failure
[ PASS    ] 0xc0deb00c::Book::get_friend_cap_success
[ PASS    ] 0xc0deb00c::Orders::scale_factor_failure
[ PASS    ] 0xc0deb00c::Book::init_book_failure_exists
[ PASS    ] 0xc0deb00c::ID::id_a_success
[ PASS    ] 0xc0deb00c::CritBit::check_len_success
[ PASS    ] 0xc0deb00c::Caps::init_caps_success
[ PASS    ] 0xc0deb00c::Caps::orders_f_c_failure
[ PASS    ] 0xc0deb00c::Book::init_book_success
[ PASS    ] 0xc0deb00c::Book::scale_factor_failure
[ PASS    ] 0xc0deb00c::ID::id_b_success
[ PASS    ] 0xc0deb00c::ID::price_success
[ PASS    ] 0xc0deb00c::ID::v_n_a_success
[ PASS    ] 0xc0deb00c::CritBit::crit_bit_success
[ PASS    ] 0xc0deb00c::CritBit::destroy_empty_fail
[ PASS    ] 0xc0deb00c::CritBit::destroy_empty_success
[ PASS    ] 0xc0deb00c::CritBit::empty_success
[ PASS    ] 0xc0deb00c::ID::v_n_b_success
[ PASS    ] 0xc0deb00c::CritBit::has_key_empty_success
[ PASS    ] 0xc0deb00c::CritBit::has_key_singleton
[ PASS    ] 0xc0deb00c::CritBit::has_key_success
[ PASS    ] 0xc0deb00c::CritBit::insert_general_failure
[ PASS    ] 0xc0deb00c::CritBit::insert_singleton_failure
[ PASS    ] 0xc0deb00c::CritBit::insert_singleton_success_l
[ PASS    ] 0xc0deb00c::CritBit::insert_singleton_success_r
[ PASS    ] 0xc0deb00c::CritBit::insert_success_1
[ PASS    ] 0xc0deb00c::CritBit::insert_success_2
[ PASS    ] 0xc0deb00c::CritBit::is_empty_success
[ PASS    ] 0xc0deb00c::CritBit::is_out_success
[ PASS    ] 0xc0deb00c::CritBit::is_set_success
[ PASS    ] 0xc0deb00c::CritBit::length_success
[ PASS    ] 0xc0deb00c::CritBit::max_key_failure_empty
[ PASS    ] 0xc0deb00c::CritBit::max_key_success
[ PASS    ] 0xc0deb00c::CritBit::min_key_failure_empty
[ PASS    ] 0xc0deb00c::CritBit::min_key_success
[ PASS    ] 0xc0deb00c::CritBit::o_v_success
[ PASS    ] 0xc0deb00c::CritBit::out_c_success
[ PASS    ] 0xc0deb00c::CritBit::pop_failure_empty
[ PASS    ] 0xc0deb00c::CritBit::pop_general_failure_no_key
[ PASS    ] 0xc0deb00c::CritBit::pop_general_success_1
[ PASS    ] 0xc0deb00c::CritBit::pop_general_success_2
[ PASS    ] 0xc0deb00c::CritBit::pop_singleton_failure
[ PASS    ] 0xc0deb00c::CritBit::pop_singleton_success
[ PASS    ] 0xc0deb00c::CritBit::singleton_success
[ PASS    ] 0xc0deb00c::CritBit::stitch_swap_remove_i_l
[ PASS    ] 0xc0deb00c::CritBit::stitch_swap_remove_i_r
[ PASS    ] 0xc0deb00c::CritBit::stitch_swap_remove_o_l
[ PASS    ] 0xc0deb00c::CritBit::stitch_swap_remove_o_r
[ PASS    ] 0xc0deb00c::CritBit::stitch_swap_remove_r_i
[ PASS    ] 0xc0deb00c::CritBit::stitch_swap_remove_r_o
[ PASS    ] 0xc0deb00c::CritBit::traverse_demo
[ PASS    ] 0xc0deb00c::CritBit::traverse_pop_success
[ PASS    ] 0xc0deb00c::CritBit::u_failure
[ PASS    ] 0xc0deb00c::CritBit::u_success

Failed coverage deserialization

(econia) econia % af-cli package coverage summary
Error: Error deserializing coverage map

Stack backtrace:
   0: backtrace::backtrace::trace
   1: anyhow::backtrace::capture::Backtrace::capture
   2: anyhow::error::<impl anyhow::Error>::msg
   3: move_coverage::coverage_map::CoverageMap::from_binary_file
   4: move_cli::package::cli::CoverageSummaryOptions::handle_command
   5: move_cli::package::cli::handle_package_commands
   6: move_cli::run_cli
   7: af_cli::main
   8: std::sys_common::backtrace::__rust_begin_short_backtrace
   9: std::rt::lang_start::{{closure}}
  10: std::rt::lang_start_internal
  11: _main

Jul 08 '22 17:07 alnoki

aptos-core aptos-core copied to clipboard

Broken coverage testing in af-cli

Overview

Reproducible behavior

Test execution failure

Failed coverage deserialization

aptos-core
aptos-core copied to clipboard