Small refactor to start making the difference between failure/non-failure cases more explicit.

Sep 09 '22 16:09 janezpodhostnik

Codecov Report

Merging #3202 (e66d53e) into master (527b2c8) will decrease coverage by 0.82%. The diff coverage is n/a.

@@            Coverage Diff             @@
##           master    #3202      +/-   ##
==========================================
- Coverage   53.99%   53.17%   -0.83%     
==========================================
  Files         743      654      -89     
  Lines       68399    57872   -10527     
==========================================
- Hits        36933    30771    -6162     
+ Misses      28415    24559    -3856     
+ Partials     3051     2542     -509

Flag	Coverage Δ
unittests	`53.17% <ø> (-0.83%)`	:arrow_down:

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files	Coverage Δ
...s/hotstuff/votecollector/staking_vote_processor.go	`83.87% <0.00%> (-3.23%)`	:arrow_down:
consensus/hotstuff/eventloop/event_loop.go	`74.82% <0.00%> (-1.37%)`	:arrow_down:
engine/collection/synchronization/engine.go	`68.97% <0.00%> (ø)`
fvm/fvm.go
fvm/transaction.go
fvm/transactionInvoker.go
fvm/transactionSequenceNum.go
fvm/transactionVerifier.go
fvm/bootstrap.go
fvm/environment/transaction_info.go
... and 83 more

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

Sep 09 '22 16:09 codecov-commenter

FVM Benchstat comparison

This branch with compared with the base branch onflow:master commit 527b2c891358e34d5d6e881c3a056585a7e2c01b

The command (for i in {1..10}; do go test ./fvm ./engine/execution/computation --bench . --tags relic -shuffle=on --benchmem --run ^$; done) was used.

Collapsed results for better readability

	old.txt	new.txt
	time/op		delta
pkg:github.com/onflow/flow-go/fvm goos:linux goarch:amd64
RuntimeNFTBatchTransfer-2	157ms ± 4%	156ms ± 5%	~	(p=0.515 n=8+10)
RuntimeTransaction/reference_tx-2	40.9ms ±13%	41.2ms ± 8%	~	(p=0.447 n=9+10)
RuntimeTransaction/convert_int_to_string-2	44.9ms ±20%	43.0ms ± 7%	~	(p=0.579 n=10+10)
RuntimeTransaction/convert_int_to_string_and_concatenate_it-2	44.7ms ± 7%	46.3ms ±12%	~	(p=0.258 n=9+9)
RuntimeTransaction/get_signer_address-2	40.8ms ± 6%	42.8ms ±17%	~	(p=0.165 n=10+10)
RuntimeTransaction/get_public_account-2	46.4ms ± 4%	46.2ms ± 5%	~	(p=0.963 n=8+9)
RuntimeTransaction/get_account_and_get_balance-2	501ms ± 4%	498ms ± 6%	~	(p=0.481 n=10+10)
RuntimeTransaction/get_account_and_get_available_balance-2	473ms ± 7%	460ms ± 5%	~	(p=0.133 n=9+10)
RuntimeTransaction/get_account_and_get_storage_used-2	50.4ms ±13%	49.0ms ± 8%	~	(p=0.353 n=10+10)
RuntimeTransaction/get_account_and_get_storage_capacity-2	420ms ± 5%	419ms ±11%	~	(p=0.684 n=10+10)
RuntimeTransaction/get_signer_vault-2	54.1ms ± 2%	53.0ms ±10%	~	(p=0.780 n=9+10)
RuntimeTransaction/get_signer_receiver-2	72.8ms ± 7%	73.2ms ± 6%	~	(p=0.912 n=10+10)
RuntimeTransaction/transfer_tokens-2	342ms ± 3%	344ms ± 1%	~	(p=0.161 n=9+9)
RuntimeTransaction/load_and_save_empty_string_on_signers_address-2	52.1ms ± 8%	52.6ms ± 6%	~	(p=0.853 n=10+10)
RuntimeTransaction/load_and_save_long_string_on_signers_address-2	111ms ± 6%	110ms ± 3%	~	(p=0.182 n=10+9)
RuntimeTransaction/create_new_account-2	1.16s ± 5%	1.15s ± 9%	~	(p=0.631 n=10+10)
RuntimeTransaction/call_empty_contract_function-2	45.4ms ±12%	45.6ms ± 6%	~	(p=0.481 n=10+10)
RuntimeTransaction/emit_event-2	67.3ms ± 7%	67.9ms ± 9%	~	(p=0.971 n=10+10)
RuntimeTransaction/borrow_array_from_storage-2	190ms ± 6%	189ms ± 3%	~	(p=0.796 n=10+10)
RuntimeTransaction/copy_array_from_storage-2	197ms ± 4%	196ms ± 3%	~	(p=0.661 n=9+10)
pkg:github.com/onflow/flow-go/engine/execution/computation goos:linux goarch:amd64
ComputeBlock/16/cols/128/txes-2	7.39s ± 3%	7.36s ± 2%	~	(p=0.796 n=10+10)

	alloc/op		delta
pkg:github.com/onflow/flow-go/fvm goos:linux goarch:amd64
RuntimeTransaction/get_account_and_get_available_balance-2	123MB ± 3%	125MB ± 2%	+2.02%	(p=0.001 n=9+10)
RuntimeNFTBatchTransfer-2	56.4MB ± 5%	56.2MB ± 3%	~	(p=0.912 n=10+10)
RuntimeTransaction/reference_tx-2	36.8MB ± 9%	37.6MB ± 8%	~	(p=0.247 n=10+10)
RuntimeTransaction/convert_int_to_string-2	37.5MB ±10%	37.5MB ± 9%	~	(p=0.912 n=10+10)
RuntimeTransaction/convert_int_to_string_and_concatenate_it-2	37.2MB ± 7%	38.7MB ± 8%	~	(p=0.143 n=10+10)
RuntimeTransaction/get_signer_address-2	36.4MB ± 7%	38.2MB ± 7%	~	(p=0.052 n=10+10)
RuntimeTransaction/get_public_account-2	38.6MB ± 6%	38.8MB ± 7%	~	(p=0.684 n=10+10)
RuntimeTransaction/get_account_and_get_balance-2	141MB ± 3%	142MB ± 2%	~	(p=0.393 n=10+10)
RuntimeTransaction/get_account_and_get_storage_used-2	39.0MB ± 8%	38.5MB ± 9%	~	(p=0.353 n=10+10)
RuntimeTransaction/get_account_and_get_storage_capacity-2	118MB ± 3%	118MB ± 3%	~	(p=0.971 n=10+10)
RuntimeTransaction/get_signer_vault-2	39.6MB ± 5%	39.5MB ± 8%	~	(p=1.000 n=9+10)
RuntimeTransaction/get_signer_receiver-2	42.3MB ± 6%	43.6MB ± 7%	~	(p=0.247 n=10+10)
RuntimeTransaction/transfer_tokens-2	90.3MB ± 5%	89.3MB ± 7%	~	(p=0.739 n=10+10)
RuntimeTransaction/load_and_save_empty_string_on_signers_address-2	40.0MB ± 9%	39.6MB ± 8%	~	(p=0.529 n=10+10)
RuntimeTransaction/load_and_save_long_string_on_signers_address-2	58.2MB ± 4%	58.9MB ± 4%	~	(p=0.481 n=10+10)
RuntimeTransaction/create_new_account-2	202MB ± 0%	204MB ± 3%	~	(p=0.237 n=8+10)
RuntimeTransaction/call_empty_contract_function-2	38.2MB ±10%	37.8MB ± 5%	~	(p=0.971 n=10+10)
RuntimeTransaction/emit_event-2	42.4MB ± 6%	42.5MB ± 6%	~	(p=0.905 n=10+9)
RuntimeTransaction/borrow_array_from_storage-2	71.2MB ± 4%	70.3MB ± 5%	~	(p=0.684 n=10+10)
RuntimeTransaction/copy_array_from_storage-2	84.0MB ± 3%	84.1MB ± 2%	~	(p=0.853 n=10+10)
pkg:github.com/onflow/flow-go/engine/execution/computation goos:linux goarch:amd64
ComputeBlock/16/cols/128/txes-2	1.32GB ± 1%	1.32GB ± 0%	~	(p=0.968 n=10+9)

	allocs/op		delta
pkg:github.com/onflow/flow-go/fvm goos:linux goarch:amd64
RuntimeNFTBatchTransfer-2	297k ± 0%	297k ± 1%	~	(p=0.649 n=8+10)
RuntimeTransaction/reference_tx-2	89.5k ± 0%	89.5k ± 0%	~	(p=0.126 n=10+10)
RuntimeTransaction/convert_int_to_string-2	104k ± 0%	104k ± 0%	~	(p=0.591 n=10+10)
RuntimeTransaction/convert_int_to_string_and_concatenate_it-2	118k ± 0%	118k ± 0%	~	(p=0.824 n=9+10)
RuntimeTransaction/get_signer_address-2	94.7k ± 0%	94.7k ± 0%	~	(p=0.065 n=10+10)
RuntimeTransaction/get_public_account-2	118k ± 0%	118k ± 0%	~	(p=0.304 n=10+10)
RuntimeTransaction/get_account_and_get_balance-2	1.71M ± 0%	1.71M ± 0%	~	(p=0.165 n=10+10)
RuntimeTransaction/get_account_and_get_available_balance-2	1.58M ± 0%	1.58M ± 0%	~	(p=0.988 n=10+10)
RuntimeTransaction/get_account_and_get_storage_used-2	139k ± 0%	139k ± 0%	~	(p=0.363 n=10+10)
RuntimeTransaction/get_account_and_get_storage_capacity-2	1.42M ± 0%	1.42M ± 0%	~	(p=0.345 n=10+9)
RuntimeTransaction/get_signer_vault-2	142k ± 0%	142k ± 0%	~	(p=0.744 n=8+10)
RuntimeTransaction/get_signer_receiver-2	222k ± 0%	222k ± 0%	~	(p=0.389 n=10+9)
RuntimeTransaction/transfer_tokens-2	971k ± 0%	971k ± 0%	~	(p=1.000 n=9+9)
RuntimeTransaction/load_and_save_empty_string_on_signers_address-2	140k ± 0%	140k ± 0%	~	(p=0.618 n=10+10)
RuntimeTransaction/create_new_account-2	2.80M ± 0%	2.80M ± 0%	~	(p=0.189 n=7+8)
RuntimeTransaction/call_empty_contract_function-2	106k ± 0%	106k ± 0%	~	(p=0.196 n=10+10)
RuntimeTransaction/emit_event-2	151k ± 0%	151k ± 0%	~	(p=0.795 n=9+10)
RuntimeTransaction/borrow_array_from_storage-2	379k ± 0%	379k ± 0%	~	(p=0.592 n=10+10)
RuntimeTransaction/copy_array_from_storage-2	335k ± 0%	335k ± 0%	~	(p=0.796 n=10+9)
pkg:github.com/onflow/flow-go/engine/execution/computation goos:linux goarch:amd64
ComputeBlock/16/cols/128/txes-2	21.0M ± 0%	21.0M ± 0%	~	(p=0.247 n=10+10)
pkg:github.com/onflow/flow-go/fvm goos:linux goarch:amd64
RuntimeTransaction/load_and_save_long_string_on_signers_address-2	242k ± 0%	242k ± 0%	−0.01%	(p=0.030 n=10+10)

	computation		delta
pkg:github.com/onflow/flow-go/fvm goos:linux goarch:amd64
RuntimeTransaction/reference_tx-2	202 ± 0%	202 ± 0%	~	(all equal)
RuntimeTransaction/convert_int_to_string-2	402 ± 0%	402 ± 0%	~	(all equal)
RuntimeTransaction/convert_int_to_string_and_concatenate_it-2	502 ± 0%	502 ± 0%	~	(all equal)
RuntimeTransaction/get_signer_address-2	302 ± 0%	302 ± 0%	~	(all equal)
RuntimeTransaction/get_public_account-2	402 ± 0%	402 ± 0%	~	(all equal)
RuntimeTransaction/get_account_and_get_balance-2	1.00k ± 0%	1.00k ± 0%	~	(all equal)
RuntimeTransaction/get_account_and_get_available_balance-2	2.60k ± 0%	2.60k ± 0%	~	(all equal)
RuntimeTransaction/get_account_and_get_storage_used-2	402 ± 0%	402 ± 0%	~	(all equal)
RuntimeTransaction/get_account_and_get_storage_capacity-2	1.30k ± 0%	1.30k ± 0%	~	(all equal)
RuntimeTransaction/get_signer_vault-2	402 ± 0%	402 ± 0%	~	(all equal)
RuntimeTransaction/get_signer_receiver-2	602 ± 0%	602 ± 0%	~	(all equal)
RuntimeTransaction/transfer_tokens-2	3.50k ± 0%	3.50k ± 0%	~	(all equal)
RuntimeTransaction/load_and_save_empty_string_on_signers_address-2	602 ± 0%	602 ± 0%	~	(all equal)
RuntimeTransaction/load_and_save_long_string_on_signers_address-2	602 ± 0%	602 ± 0%	~	(all equal)
RuntimeTransaction/create_new_account-2	202 ± 0%	202 ± 0%	~	(all equal)
RuntimeTransaction/call_empty_contract_function-2	402 ± 0%	402 ± 0%	~	(all equal)
RuntimeTransaction/emit_event-2	602 ± 0%	602 ± 0%	~	(all equal)
RuntimeTransaction/borrow_array_from_storage-2	2.60k ± 0%	2.60k ± 0%	~	(all equal)
RuntimeTransaction/copy_array_from_storage-2	2.60k ± 0%	2.60k ± 0%	~	(all equal)

	interactions		delta
pkg:github.com/onflow/flow-go/fvm goos:linux goarch:amd64
RuntimeTransaction/reference_tx-2	44.4k ± 0%	44.4k ± 0%	~	(all equal)
RuntimeTransaction/convert_int_to_string-2	44.4k ± 0%	44.4k ± 0%	~	(all equal)
RuntimeTransaction/convert_int_to_string_and_concatenate_it-2	44.4k ± 0%	44.4k ± 0%	~	(all equal)
RuntimeTransaction/get_signer_address-2	44.4k ± 0%	44.4k ± 0%	~	(all equal)
RuntimeTransaction/get_public_account-2	44.4k ± 0%	44.4k ± 0%	~	(all equal)
RuntimeTransaction/get_account_and_get_balance-2	16.8M ± 0%	16.8M ± 0%	~	(all equal)
RuntimeTransaction/get_account_and_get_available_balance-2	5.28M ± 0%	5.28M ± 0%	~	(all equal)
RuntimeTransaction/get_account_and_get_storage_used-2	48.0k ± 0%	48.0k ± 0%	~	(all equal)
RuntimeTransaction/get_account_and_get_storage_capacity-2	5.27M ± 0%	5.27M ± 0%	~	(all equal)
RuntimeTransaction/get_signer_vault-2	44.7k ± 0%	44.7k ± 0%	~	(all equal)
RuntimeTransaction/get_signer_receiver-2	45.0k ± 0%	45.0k ± 0%	~	(all equal)
RuntimeTransaction/transfer_tokens-2	45.8k ± 0%	45.8k ± 0%	~	(all equal)
RuntimeTransaction/load_and_save_empty_string_on_signers_address-2	44.8k ± 0%	44.8k ± 0%	~	(all equal)
RuntimeTransaction/load_and_save_long_string_on_signers_address-2	49.7k ± 0%	49.7k ± 0%	~	(all equal)
RuntimeTransaction/create_new_account-2	8.53M ± 0%	8.53M ± 0%	~	(all equal)
RuntimeTransaction/call_empty_contract_function-2	44.6k ± 0%	44.6k ± 0%	~	(all equal)
RuntimeTransaction/emit_event-2	44.6k ± 0%	44.6k ± 0%	~	(all equal)
RuntimeTransaction/borrow_array_from_storage-2	49.8k ± 0%	49.8k ± 0%	~	(all equal)
RuntimeTransaction/copy_array_from_storage-2	49.8k ± 0%	49.8k ± 0%	~	(all equal)

	us/tx		delta
pkg:github.com/onflow/flow-go/engine/execution/computation goos:linux goarch:amd64
ComputeBlock/16/cols/128/txes-2	3.61k ± 3%	3.59k ± 2%	~	(p=0.796 n=10+10)

Sep 09 '22 16:09 github-actions[bot]

Since coded error this is much less needed, and would also need to be rewritten

Oct 19 '22 15:10 janezpodhostnik

flow-go
flow-go copied to clipboard

[FVM] Make failure more explicit

Codecov Report

FVM Benchstat comparison

flow-go flow-go copied to clipboard

[FVM] Make failure more explicit

Codecov Report

FVM Benchstat comparison

flow-go
flow-go copied to clipboard