Storage Iteration functions for accounts

[dsainati1]

Part of https://github.com/onflow/cadence/issues/208

Description

This adds three new functions to the AuthAccount object forEachPublic, forEachPrivate and forEachStored (as well as just forEachPublic to PublicAccount) that iterate over all the paths (and the types of the objects stored there) in the respective domains of the account.

Each of these functions takes a function parameter of type (X, Type): Bool (where X is the type of the path being iterated over by the function in question (PublicPath, PrivatePath and StoragePath respectively)), which is applied to every element stored in that account under the domain in question. The first parameter holds the path of the element, and the second holds its runtime type.

These paths and types are loaded from storage one by one, to avoid loading the entirety of storage into memory in the case of large accounts.

The specific order in which the objects are iterated over is undefined, as is the behavior when a path is added or removed from storage, whether by linking, unlinking, saving or loading. The behavior is, however, deterministic; we make no guarantees about what we do but we do the same thing every time.

---

• [X] Targeted PR against master branch
• [X] Linked to Github issue with discussion and accepted design OR link to spec that describes this work
• [X] Code follows the standards mentioned here
• [X] Updated relevant documentation
• [X] Re-reviewed Files changed in the Github PR explorer
• [X] Added appropriate labels

[bluesign]

These paths and types are loaded from storage one by one.

Why do we load full value here ? Shouldn't we load type info only ? This can cause big usability problems.

[dsainati1]

We do load only the runtime type info here, the actual stored value itself is not loaded.

[codecov[bot]]

Codecov Report

Merging #1851 (13f2999) into master (2982f32) will increase coverage by 0.03%. The diff coverage is 96.26%.

@@            Coverage Diff             @@
##           master    #1851      +/-   ##
==========================================
+ Coverage   77.78%   77.82%   +0.03%     
==========================================
  Files         294      294              
  Lines       62403    62508     +105     
==========================================
+ Hits        48540    48644     +104     
  Misses      12137    12137              
- Partials     1726     1727       +1     

Flag	Coverage Δ
unittests	77.82% <96.26%> (+0.03%)	:arrow_up:

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files	Coverage Δ
runtime/interpreter/interpreter.go	89.63% <89.74%> (+<0.01%)	:arrow_up:
runtime/interpreter/account.go	89.69% <100.00%> (+0.80%)	:arrow_up:
runtime/interpreter/value.go	72.14% <100.00%> (+0.02%)	:arrow_up:
runtime/sema/authaccount_type.go	100.00% <100.00%> (ø)
runtime/sema/publicaccount_type.go	100.00% <100.00%> (ø)
runtime/interpreter/storage.go	69.04% <0.00%> (-1.59%)	:arrow_down:
runtime/sema/check_event_declaration.go	100.00% <0.00%> (+5.76%)	:arrow_up:

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

[github-actions[bot]]

Cadence Benchstat comparison

This branch with compared with the base branch onflow:master commit 2982f325bbba9b25613fac43f331cf66ce1b49cb The command for i in {1..N}; do go test ./... -run=XXX -bench=. -benchmem -shuffle=on; done was used. Bench tests were run a total of 7 times on each branch.

Collapsed results for better readability

	old.txt	new.txt
	time/op		delta
CheckContractInterfaceFungibleTokenConformance-2	136µs ± 1%	137µs ± 2%	~	(p=0.209 n=7+7)
ContractInterfaceFungibleToken-2	40.2µs ± 1%	40.1µs ± 1%	~	(p=0.456 n=7+7)
InterpretRecursionFib-2	3.59ms ± 1%	3.55ms ± 6%	~	(p=0.165 n=7+7)
NewInterpreter/new_interpreter-2	1.18µs ± 1%	1.18µs ± 1%	~	(p=0.312 n=6+7)
NewInterpreter/new_sub-interpreter-2	2.43µs ± 1%	2.44µs ± 2%	~	(p=0.731 n=6+7)
ParseArray-2	8.62ms ± 1%	8.71ms ± 2%	~	(p=0.240 n=6+6)
ParseDeploy/byte_array-2	13.0ms ± 2%	13.7ms ± 6%	+4.93%	(p=0.017 n=7+7)
ParseDeploy/decode_hex-2	1.14ms ± 1%	1.15ms ± 2%	~	(p=0.138 n=6+7)
ParseFungibleToken/With_memory_metering-2	210µs ± 2%	206µs ± 1%	−1.65%	(p=0.022 n=7+6)
ParseFungibleToken/Without_memory_metering-2	159µs ± 1%	161µs ± 2%	+1.27%	(p=0.015 n=6+6)
ParseInfix-2	7.77µs ± 2%	7.78µs ± 1%	~	(p=0.589 n=6+6)
QualifiedIdentifierCreation/One_level-2	2.68ns ± 0%	2.35ns ± 0%	−12.45%	(p=0.001 n=7+7)
QualifiedIdentifierCreation/Three_levels-2	138ns ± 0%	137ns ± 2%	~	(p=0.181 n=7+6)
RuntimeFungibleTokenTransfer-2	1.03ms ±21%	0.85ms ± 2%	~	(p=0.181 n=7+6)
RuntimeResourceDictionaryValues-2	6.42ms ± 2%	6.40ms ± 3%	~	(p=0.805 n=7+7)
ValueIsSubtypeOfSemaType-2	90.1ns ± 1%	89.3ns ± 3%	~	(p=0.534 n=6+7)
	alloc/op		delta
CheckContractInterfaceFungibleTokenConformance-2	62.5kB ± 0%	62.5kB ± 0%	~	(p=1.000 n=7+7)
ContractInterfaceFungibleToken-2	24.4kB ± 0%	24.4kB ± 0%	~	(p=0.462 n=7+7)
InterpretRecursionFib-2	1.51MB ± 0%	1.51MB ± 0%	~	(p=0.294 n=7+6)
NewInterpreter/new_interpreter-2	928B ± 0%	928B ± 0%	~	(all equal)
NewInterpreter/new_sub-interpreter-2	1.46kB ± 0%	1.46kB ± 0%	~	(all equal)
ParseArray-2	2.79MB ± 2%	2.82MB ± 2%	~	(p=0.079 n=7+7)
ParseDeploy/byte_array-2	4.33MB ± 0%	4.42MB ± 3%	~	(p=0.836 n=6+7)
ParseDeploy/decode_hex-2	214kB ± 0%	214kB ± 0%	~	(p=0.560 n=7+7)
ParseFungibleToken/With_memory_metering-2	36.3kB ± 0%	36.3kB ± 0%	~	(p=0.640 n=7+7)
ParseFungibleToken/Without_memory_metering-2	36.3kB ± 0%	36.3kB ± 0%	~	(p=1.000 n=7+7)
ParseInfix-2	2.17kB ± 0%	2.17kB ± 0%	~	(p=0.709 n=7+7)
QualifiedIdentifierCreation/One_level-2	0.00B	0.00B	~	(all equal)
QualifiedIdentifierCreation/Three_levels-2	64.0B ± 0%	64.0B ± 0%	~	(all equal)
RuntimeFungibleTokenTransfer-2	185kB ± 0%	186kB ± 0%	+0.52%	(p=0.001 n=7+7)
RuntimeResourceDictionaryValues-2	2.23MB ± 0%	2.23MB ± 0%	~	(p=0.318 n=7+7)
ValueIsSubtypeOfSemaType-2	48.0B ± 0%	48.0B ± 0%	~	(all equal)
	allocs/op		delta
CheckContractInterfaceFungibleTokenConformance-2	1.03k ± 0%	1.03k ± 0%	~	(all equal)
ContractInterfaceFungibleToken-2	428 ± 0%	428 ± 0%	~	(all equal)
InterpretRecursionFib-2	33.5k ± 0%	33.5k ± 0%	~	(all equal)
NewInterpreter/new_interpreter-2	13.0 ± 0%	13.0 ± 0%	~	(all equal)
NewInterpreter/new_sub-interpreter-2	42.0 ± 0%	42.0 ± 0%	~	(all equal)
ParseArray-2	70.0k ± 0%	70.0k ± 0%	~	(p=1.000 n=7+7)
ParseDeploy/byte_array-2	105k ± 0%	105k ± 0%	~	(p=1.000 n=7+7)
ParseDeploy/decode_hex-2	77.0 ± 0%	77.0 ± 0%	~	(all equal)
ParseFungibleToken/With_memory_metering-2	1.06k ± 0%	1.06k ± 0%	~	(all equal)
ParseFungibleToken/Without_memory_metering-2	1.06k ± 0%	1.06k ± 0%	~	(all equal)
ParseInfix-2	66.0 ± 0%	66.0 ± 0%	~	(all equal)
QualifiedIdentifierCreation/One_level-2	0.00	0.00	~	(all equal)
QualifiedIdentifierCreation/Three_levels-2	2.00 ± 0%	2.00 ± 0%	~	(all equal)
RuntimeFungibleTokenTransfer-2	3.50k ± 0%	3.52k ± 0%	+0.29%	(p=0.001 n=7+6)
RuntimeResourceDictionaryValues-2	37.1k ± 0%	37.1k ± 0%	+0.01%	(p=0.001 n=7+7)
ValueIsSubtypeOfSemaType-2	1.00 ± 0%	1.00 ± 0%	~	(all equal)