casper-node
casper-node copied to clipboard
Add instrumentation to the test suite
For each exec() call in the test suite, this PR attaches instrumentation data that will uniquely identify a given run, together with duration, gas_cost, and error and save this data into a .csv file that can be used for data analysis. This way, after modifying opcode cost table there's a way to measure real impact across all the exec runs.
Usage:
builder.exec_instrumented(exec_request, instrument!());
instrument!
macro accepts extra (key, value) pairs that will be also appended to the CSV i.e. in case there's a for loop with builder.exec
calls:
for i in 0..100 {
let exec_request = make_request(i);
builder.exec_instrumented(exec_request, instrumented!(i));
}
This should append additional fields to the CSV row: i, 0
etc.
Sample output in execution_results.txt
:
module_name,file,line,function,duration_ns,gas,error
casper_engine_tests::test::deploy::stored_contracts,execution_engine_testing/tests/src/test/deploy/stored_contracts.rs,161,should_fail_if_calling_non_existent_entry_point,3363,13663671535,
casper_engine_tests::test::deploy::stored_contracts,execution_engine_testing/tests/src/test/deploy/stored_contracts.rs,196,should_fail_if_calling_non_existent_entry_point,283,2500000000,No such method: electric-boogaloo
casper_engine_tests::test::deploy::stored_contracts,execution_engine_testing/tests/src/test/deploy/stored_contracts.rs,645,should_fail_payment_stored_at_hash_with_incompatible_major_version,3076,13663671535,
casper_engine_tests::test::deploy::stored_contracts,execution_engine_testing/tests/src/test/deploy/stored_contracts.rs,696,should_fail_payment_stored_at_hash_with_incompatible_major_version,317,2500000000,Incompatible protocol major version. Expected version 2 but actual version is 1