cadence icon indicating copy to clipboard operation
cadence copied to clipboard

Document setup() and tearDown() functions in test-framework

Open SupunS opened this issue 3 years ago • 2 comments


  • [x] Targeted PR against master branch
  • [x] Linked to Github issue with discussion and accepted design OR link to spec that describes this work
  • [x] Code follows the standards mentioned here
  • [x] Updated relevant documentation
  • [x] Re-reviewed Files changed in the Github PR explorer
  • [x] Added appropriate labels

SupunS avatar Oct 07 '22 21:10 SupunS

Codecov Report

Merging #2043 (ffb19e6) into master (d8b6241) will not change coverage. The diff coverage is n/a.

@@           Coverage Diff           @@
##           master    #2043   +/-   ##
=======================================
  Coverage   77.61%   77.61%           
=======================================
  Files         303      303           
  Lines       63164    63164           
=======================================
  Hits        49024    49024           
  Misses      12394    12394           
  Partials     1746     1746           
Flag Coverage Δ
unittests 77.61% <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

codecov[bot] avatar Oct 07 '22 21:10 codecov[bot]

Cadence Benchstat comparison

This branch with compared with the base branch onflow:master commit d8b6241606b9dd78a5ca5d9b646dfa27f7842037 The command for i in {1..N}; do go test ./... -run=XXX -bench=. -benchmem -shuffle=on; done was used. Bench tests were run a total of 7 times on each branch.

Collapsed results for better readability

old.txtnew.txt
time/opdelta
CheckContractInterfaceFungibleTokenConformance-2123µs ± 5%122µs ± 7%~(p=0.710 n=7+7)
ContractInterfaceFungibleToken-242.3µs ± 4%41.8µs ± 1%~(p=0.445 n=7+6)
InterpretRecursionFib-22.41ms ± 3%2.37ms ± 3%~(p=0.128 n=7+7)
NewInterpreter/new_interpreter-21.31µs ± 7%1.22µs ± 1%−7.14%(p=0.001 n=7+6)
NewInterpreter/new_sub-interpreter-2712ns ± 5%679ns ± 1%−4.62%(p=0.035 n=7+6)
ParseArray-28.59ms ± 7%8.55ms ± 5%~(p=0.805 n=7+7)
ParseDeploy/byte_array-213.1ms ± 6%12.9ms ± 4%~(p=0.620 n=7+7)
ParseDeploy/decode_hex-21.23ms ± 1%1.24ms ± 1%~(p=0.209 n=7+7)
ParseFungibleToken/With_memory_metering-2193µs ± 1%193µs ± 3%~(p=0.699 n=6+6)
ParseFungibleToken/Without_memory_metering-2156µs ± 2%155µs ± 2%~(p=0.805 n=7+7)
ParseInfix-27.33µs ± 2%7.26µs ± 2%~(p=0.209 n=7+7)
QualifiedIdentifierCreation/One_level-22.35ns ± 1%2.35ns ± 0%~(p=1.000 n=7+7)
QualifiedIdentifierCreation/Three_levels-2139ns ± 1%138ns ± 1%~(p=0.097 n=7+6)
RuntimeFungibleTokenTransfer-2657µs ±27%593µs ±29%~(p=0.097 n=7+7)
RuntimeResourceDictionaryValues-25.35ms ± 4%5.23ms ± 2%~(p=0.128 n=7+7)
RuntimeScriptNoop-218.3µs ±35%17.8µs ±37%~(p=0.318 n=7+7)
ValueIsSubtypeOfSemaType-292.6ns ± 2%92.7ns ± 1%~(p=0.780 n=7+7)
 
alloc/opdelta
CheckContractInterfaceFungibleTokenConformance-251.0kB ± 0%51.0kB ± 0%~(p=1.000 n=7+7)
ContractInterfaceFungibleToken-224.2kB ± 0%24.2kB ± 0%~(all equal)
InterpretRecursionFib-21.00MB ± 0%1.00MB ± 0%~(p=0.213 n=7+7)
NewInterpreter/new_interpreter-2848B ± 0%848B ± 0%~(all equal)
NewInterpreter/new_sub-interpreter-2264B ± 0%264B ± 0%~(all equal)
ParseArray-22.73MB ± 3%2.78MB ± 4%~(p=0.318 n=7+7)
ParseDeploy/byte_array-24.24MB ± 3%4.18MB ± 3%~(p=0.209 n=7+7)
ParseDeploy/decode_hex-2214kB ± 0%214kB ± 0%~(p=0.245 n=7+7)
ParseFungibleToken/With_memory_metering-229.4kB ± 0%29.4kB ± 0%~(p=1.000 n=7+7)
ParseFungibleToken/Without_memory_metering-229.4kB ± 0%29.4kB ± 0%~(p=0.085 n=6+7)
ParseInfix-21.93kB ± 0%1.93kB ± 0%~(p=0.633 n=7+7)
QualifiedIdentifierCreation/One_level-20.00B 0.00B ~(all equal)
QualifiedIdentifierCreation/Three_levels-264.0B ± 0%64.0B ± 0%~(all equal)
RuntimeFungibleTokenTransfer-2102kB ± 1%102kB ± 0%~(p=0.366 n=7+6)
RuntimeResourceDictionaryValues-22.27MB ± 0%2.27MB ± 0%~(p=0.535 n=7+7)
RuntimeScriptNoop-28.83kB ± 0%8.88kB ± 1%~(p=0.137 n=7+7)
ValueIsSubtypeOfSemaType-248.0B ± 0%48.0B ± 0%~(all equal)
 
allocs/opdelta
CheckContractInterfaceFungibleTokenConformance-2915 ± 0%915 ± 0%~(all equal)
ContractInterfaceFungibleToken-2436 ± 0%436 ± 0%~(all equal)
InterpretRecursionFib-218.9k ± 0%18.9k ± 0%~(all equal)
NewInterpreter/new_interpreter-214.0 ± 0%14.0 ± 0%~(all equal)
NewInterpreter/new_sub-interpreter-25.00 ± 0%5.00 ± 0%~(all equal)
ParseArray-269.6k ± 0%69.6k ± 0%~(p=0.767 n=6+7)
ParseDeploy/byte_array-2104k ± 0%104k ± 0%~(p=1.000 n=7+7)
ParseDeploy/decode_hex-275.0 ± 0%75.0 ± 0%~(all equal)
ParseFungibleToken/With_memory_metering-21.00k ± 0%1.00k ± 0%~(all equal)
ParseFungibleToken/Without_memory_metering-21.00k ± 0%1.00k ± 0%~(all equal)
ParseInfix-260.0 ± 0%60.0 ± 0%~(all equal)
QualifiedIdentifierCreation/One_level-20.00 0.00 ~(all equal)
QualifiedIdentifierCreation/Three_levels-22.00 ± 0%2.00 ± 0%~(all equal)
RuntimeFungibleTokenTransfer-22.06k ± 0%2.06k ± 0%~(all equal)
RuntimeResourceDictionaryValues-236.9k ± 0%36.9k ± 0%~(p=0.540 n=7+7)
RuntimeScriptNoop-2145 ± 0%145 ± 0%~(all equal)
ValueIsSubtypeOfSemaType-21.00 ± 0%1.00 ± 0%~(all equal)
 

github-actions[bot] avatar Oct 07 '22 21:10 github-actions[bot]