Reuse interpreted programs within a transaction

[janezpodhostnik]

Issue To Be Solved

This relates to this slack discussion.

Exploring the profile of execution an empty transaction (transaction that does basically nothing) I noticed that EsureLoaded is called multiple times during one transaction and is using a significant amount of time. Most of the time within EsureLoaded is spent inside Interpreter.Interpret building interpretations.

Ensure loaded is used in the three different parts of the trasanction:

• invoke users transaction
• deduct fees
• check accounts storage limits

In the scenario profiled above, all programs should be in the programs cache, very little time is needed to retrieve a program from the programs cache.

Deduct fees and check accounts storage limits share a lot of contracts (FlowToken, FungibleToken, FlowServiceAccount) and since those are commonly used invoke users transaction might have needed them as well. So It would be beneficial to reuse the interpretation between these three parts of the trasnaction.

The reuse would happen only within a transaction, and would be dropped if a revert happened (failed fee deduction or storage check or the invocation itself).

After some more digging I managed to create a PoC:

• https://github.com/onflow/cadence/compare/v0.27.0...janez/poc-interpreter-shared-state-reuse
• https://github.com/onflow/flow-go/compare/tonyz/fvm_setting_prefetch...janez/poc-interpreter-shared-state-reuse

... and it looks like this might lead to a 15% speedup in execution times.

Suggested Solution

There are two different use cases for NewInterpreter One is when program is not nil (transactions, scripts) and one where it is (contract function invocation, get stored value). I think those two cases should be treated separately on the Environment interface. With the case where the program is nil, you should be able to specify an existing interpreter (or better yet an interpreter.sharedState) when you have one. The FVM could grab the interpreter (or shared state) created during transaction invocation, and use it for calls to contract function invocations and get stored value.

The problem is that the interpreter (and also shared state) is not stateless, and I am not sure if calling CommitStorage on it multiple times is safe.

[turbolent]

@janezpodhostnik I had a look at https://github.com/onflow/cadence/compare/v0.27.0...janez/poc-interpreter-shared-state-reuse: Looks good! Maybe open a (draft) PR and we can iterate on it. It looks like this addresses the case where there is no interpreter? Is that what the 15% speedup is for?

[turbolent]

@janezpodhostnik As far as I understand the current approach, the idea is:

• Cadence "pushes" an interpreter to FVM with SetInterpreter, basically Cadence suggests to FVM to hold on to an interpreter
• In turn, FVM can provide an interpreter it was given to Cadence in a subsequent use of the runtime – by having Cadence "pull" from FVM using GetInterpreter. FVM maybe return an interpreter that it was given using SetInterpreter

Are there any advantages to this "pull" based approach and implementing it in the interface?

We already had previously introduced the concept of "reusable runtime parts" as an "environment" (for lack of better naming, suggestions welcome). Currently, embedders of Cadence / uses of the runtime can choose to create an environment once, then keep passing it inside of the context to the runtime functions, e.g. ExecuteTransaction. Maybe we can apply this same "push" principle for the interpreter re-use?

[janezpodhostnik]

@turbolent

As far as I understand the current approach, the idea is:

Yes, that is what I was trying to illustrate

Is that what the 15% speedup is for?

The speedup is coming from fee deduction and account storage checking reusing the interpreter (and thus interpretations) from the transaction invocation itself.

It looks like this addresses the case where there is no interpreter.

Yes, I tried to make it so that you can reuse an existing interpreter, but you don't need to.

My main concern was the scope of where the interpreter can be reused:

As far as I understand the interpreter (or interpreter shared state) can be (safely) reused within the same branch of a transaction. By branch I'm referring to the fact that if the transaction fails and needs to rollback and continue to the fee deduction the interpreter should not be reused after rolling back.

If this is correct, the interpreter that the FVM is holding on to has a very different scope than the "interpreter environment". However a interpreter cache/handler/manager could be a part of the "interpreter environment" but it would need to know when a transaction branch goes out of scope, which is something only the FVM can provide.

[turbolent]

@janezpodhostnik I see, thank you for the explanation, that makes sense 👍