CLI Feature: State management

[srinjoyc]

👋   If you are interested in working on this issue, please check out the Getting Started guide on HackerEarth!

Description (Problem Statement)

The CLI should allow saving the current state and then later restarting from that point similar to how you can use git to move between states. This feature is mostly implemented in the PR branch but should be tested, extended with a test suite, and ensure the UX is great.

Experience Required

• Experience with Go
• Basic understanding of Flow blockchain

Minimum Feature Set (Acceptance Criteria)

A developer has commands to achieve the following actions:

• saving the state under a name
• restoring to the named state
• deleting a state
• overwriting a state

The state should be persistent between restarts

Extension (Optional) Feature Set

Interoperability with the Emulator GUI task.

Milestone Requirements

1. Finish the implementation and making sure it works
2. Write tests around the feature proving the functionality of it
3. Documentation of the feature

Software Requirements

Maintainability

Code should be written with best practices in mind.

Testing

Tests should cover all the code implementing this functionality.

Other Requirements

Documentation

• All the commands and the usage should be well documented.

Code standards or guidelines

• Follow the general contribution guidelines.

Judging Criteria

• For all projects, technical & high-level judging criteria apply.

Resources

• General Resource Guide
• CLI
  • Repository
  • Design guidelines
• The issue
• Branch with feature mostly implemented

[sideninja]

Hi 👋 my name is Gregor, and as a software engineer working on developer tools, I will be your point of contact for this task, so good luck, and don't hesitate to reach out to me with any problems or questions.

I'll keep an eye on your progress and will be reviewing your code.

You can comment here or find me on Discord (I'm sideninja#1970). Join the Flow Discord server if you're not there already!

[bluesign]

Hey @sideninja ,

I would love to have this feature. I like the snapshot concept, but what would you think to extend this task a bit more to instead emulator to expose subset of git functionality, it would provide a git compatibility layer on storage.

When new block committed, commit to git state and use Block information, transaction status / events etc as git commit message.

Git is already providing log, tag, branch, remote origin etc support. Also as cli command same required functionality can be achieved. (just using git as underlying version controlling)

[sideninja]

Hey @sideninja ,

I would love to have this feature. I like the snapshot concept, but what would you think to extend this task a bit more to instead emulator to expose subset of git functionality, it would provide a git compatibility layer on storage.

When new block committed, commit to git state and use Block information, transaction status / events etc as git commit message.

Git is already providing log, tag, branch, remote origin etc support. Also as cli command same required functionality can be achieved. (just using git as underlying version controlling)

That's a very interesting idea. Git could be even used as a go module in the emulator to provide this functionality. It's a bit funny because the ledger itself has all the history but the storage layer doesn't provide an easy API for this so it probably makes sense to have this kind of "layer 2" solution.

[bluesign]

yeah I first tried on ledger side, but branching is a big problem there, moving on a single like is easy, when you try to branch things are getting a bit complicated.

But with git module it is almost supporting every git operation + cli can translate cli command to git commands too. Also log is pretty nice.

[bluesign]

Team: https://www.hackerearth.com/challenges/hackathon/flip-fest/dashboard/3744a70/team/ @bluesign (Tackling solo) plan time: 2021-10-17

So I will work on this one, I would like to integrate also with https://github.com/onflow/cadence/issues/1160 so we can have a command to use in testing to load, save and switch to some snapshot.

[psiemens]

@bluesign Awesome, you're approved 😄

[bluesign]

I managed to finish first part: integrating git to emulator, committing to git on transaction executions etc.

But stuck with integrating into cadence ( as discussed in onflow/cadence#1160 )

Actually I managed to integrate by creating a newScriptInvocator and replacing vm's ScriptProcessors. ( Technically switching state in transaction would not be possible anyway )

So far I manage to create a switchState cadence built-in method, and by executing script you can change emulator state ( switching to another branch in git )

But if someone from FVM side can give some light, maybe there is a better way to do this.

[sideninja]

I managed to finish first part: integrating git to emulator, committing to git on transaction executions etc.

But stuck with integrating into cadence ( as discussed in onflow/cadence#1160 )

Actually I managed to integrate by creating a newScriptInvocator and replacing vm's ScriptProcessors. ( Technically switching state in transaction would not be possible anyway )

So far I manage to create a switchState cadence built-in method, and by executing script you can change emulator state ( switching to another branch in git )

But if someone from FVM side can give some light, maybe there is a better way to do this.

Maybe @janezpodhostnik can help with FVM.

[bluesign]

Thanks, it would be the easiest I guess. Otherwise another option would require another RPC channel.

[janezpodhostnik]

@bluesign and I had a chat, and I've also done some thinking. Here are my notes.

Creating a new ScriptInvocator and replacing the vm's ScriptProcessors is the way to go currently, and is the best solution for now and I'm totally on board with it. The thing to note is that if the ScriptInvocator changes in FVM the new ScriptInvocator in the emulator also needs to be updated (but that does not happen often).

The suggestion was that the ScriptProcessors would be able to "pass on" the predeclared values to the next ScriptProcessor, that way there would be no need to redefine the ScriptInvocator but new ScriptProcessors could just be added earlier in the pipeline to define the cadence predeclared values, which the ScriptInvocator would then use. It's definitely an interesting suggestion, that I have to look into.

It might also help if there was the possibility to add global predeclared values to the context that are appended to each script/tx processor. I have to check if that would be helpful.

I am slightly worried that "operational", emulator specific, cadence functions are being added, as that might be confusing for user when they switch to/from the emulator. Namespacing these functions would definitely help, but maybe it might be worth considering having another communication channel to/from the emulator for these kind of functionalities.

[bluesign]

After @janezpodhostnik's comment, I am leaning more towards to a different RPC for cli and emulator to talk, if there are no objections I will go that path. ( Glue code to extend go functions to emulator I will develop on a separate project, as it is almost ready )

[sideninja]

After @janezpodhostnik's comment, I am leaning more towards to a different RPC for cli and emulator to talk, if there are no objections I will go that path. ( Glue code to extend go functions to emulator I will develop on a separate project, as it is almost ready )

I think establishing another channel of communication between CLI and emulator would be good. I was thinking that channel could even be HTTP API as there is already HTTP API listening on port 8080 and handles metrics and liveness checks and that feels to me already as an API that could be extended and would be emulator management API. I could even see more developer APIs added there, think APIs specific to local usage and not supported by the network such as triggering new blocks to APIs, getting account storage, etc etc. Creating a new grpc service I fear may cause confusion, although I think separating things by communication protocol is never a good design decision in this case as there already is an HTTP API it just might be ok. What do you think?

[bluesign]

yeah I was thinking RPC as HTTP API actually, I am extending 8080 with some rest methods then. That will be very good. If you have any more ideas please share.

I am thinking below currently:

• state save/switch
• explore storage ( I have the code already in somewhere )
• emulator time drift ( same as above )
• trigger new block ( this should be easy )

[sideninja]

yeah I was thinking RPC as HTTP API actually, I am extending 8080 with some rest methods then. That will be very good. If you have any more ideas please share.

I am thinking below currently:

• state save/switch
• explore storage ( I have the code already in somewhere )
• emulator time drift ( same as above )
• trigger new block ( this should be easy )

Ohh I missread as grpc, yeah be free to implement it as RPC. Ohh wow, would love to see those as part of the emulator, specially the storage explorer is quite needed for dev, but at some later point that will be natively supported by Cadence, however until that day comes it's still great addition.

[kimcodeashian]

Good day @bluesign!

Thanks so much for all your hardwork & participation. In order to finalize winners & prepare for prize payout, we'll need the following actions from your end.

Please provide the following information by Nov 17, 2021, (in this GH Issue is fine):

1. Team Information

• Team Members Information - Github Username + Email Contact + Percentage of prize allocation (total should = 100%)
• All mentioned members MUST react to the post with a 👍 which will act as confirmation that the information is correct, or a 👎 to indicate that the information is not correct.
• We will be reaching out via e-mail

🎖IMPORTANT: We will only proceed with prize payouts once all members have confirmed with 👍 on the post.

2. Video Demo (optional)

• Please provide a 5-minute video demo to be featured & showcased in the FLIP Fest Closing Ceremonies
• Link format & Downloadable (eg. Google Drive, Vimeo)
• Content Format (Problem Statement, your work / how you solved it, final outcome)

We will be hosting Closing Ceremonies on November 23rd, 8AM PT where we'll having closing remarks from Dete & will be announcing the winners! I'll share the details here before Nov 17.

[bluesign]

@bluesign [email protected] 100%

[kimcodeashian]

Hey folks,

We've received and reviewed over 82 submissions! What an amazing community on Flow! To commemorate all the hard work done, we have finalized winners and will be announcing them during our Closing Ceremony on Nov 23rd, 8AM PT. Be sure to join us - there may be some attendance prizes & a keynote from our CTO, Dete 😉!

RSVP here so you don't miss out! See you then!