hive
hive copied to clipboard
ethereum
create a KEVM client for consensus tests
KEVM (https://github.com/kframework/evm-semantics) is an executable spec that is already capable of running test cases.
Hive has a consensus test runner, which is used to run consensus test suite against each client and generate a report dashboard: http://hivetests.ethstats.net/
The goal is to see KEVM on the hivetests dashboard, alongside the other clients. To accomplish this, we don't need to build up KEVM into a full client, but we would need an RPC server that can parse the results printed by KEVM and reply to the Hive testrunner, to answer queries like eth_getLatestBlock. These RPC responses are used by Hive to verify that the client passed the test.
Issue Status: 1. Open 2. Started 3. Submitted 4. Done
This issue now has a funding of 1000.0 DAI (1000.0 USD @ $1.0/DAI) attached to it.
- If you would like to work on this issue you can 'start work' on the Gitcoin Issue Details page.
- Want to chip in? Add your own contribution here.
- Questions? Checkout Gitcoin Help or the Gitcoin Slack
- $49,790.09 more funded OSS Work available on the Gitcoin Issue Explorer
Issue Status: 1. Open 2. Started 3. Submitted 4. Done
Workers have applied to start work.
These users each claimed they can complete the work by 4 weeks, 1 day from now. Please review their action plans below:
1) g-r-a-n-t has applied to start work (Funders only: approve worker | reject worker).
- Get Hive and KEVM running locally
- Better understand technology and desired results of ticket
- Create RPC service that can handle the queries described
- Integrate this service with Hive
Hard to estimate the time required, since Im not very familiar with this. Rough estimate if things go well: 1 week
Learn more on the Gitcoin Issue Details page.
Issue Status: 1. Open 2. Started 3. Submitted 4. Done
Work has been started.
These users each claimed they can complete the work by 3 months, 3 weeks ago. Please review their action plans below:
1) somniastellarum has been approved to start work.
Having worked on a couple hive issues before, I have a good grasp of the Go Code base. From what the issue describes, I would need to take an output (I'm guessing just a defined file) and create a server to reply to RPC calls. I would also need to create the client in Hive such that it returns the correct results.
I would need some time to define all the requirements of this server. For example, I don't know exactly how many RPC calls would be needed to be implemented, and I don't think implementing everything would be needed, just what Hive calls. Once the requirements are setup I think it should be relatively straightforward. The server isn't too hard to code. Setting up the client should be relatively straightforward through docker and some shells scripts and testing should be straightforward at that point.
Learn more on the Gitcoin Issue Details page.
Awesome, I'll get started right away. I'll start researching exactly how KEVM works. Is there any chance I could have a short 10 min call with someone (on zoom or something)? There are enough little details that I wouldn't mind a quick call just to walk through my thinking on it all so far and make sure it all makes sense.
@SomniaStellarum Hello from Gitcoin Core - are you still working on this issue? Please submit a WIP PR or comment back within the next 3 days or you will be removed from this ticket and it will be returned to an ‘Open’ status. Please let us know if you have questions!
- [x] warning (3 days)
- [ ] escalation to mods (6 days)
Funders only: Snooze warnings for 1 day | 3 days | 5 days | 10 days | 100 days
Still doing research and getting started. Could we snooze this bot as it's a longer project and for the next couple days I'll be getting up to speed?
@SomniaStellarum Hello from Gitcoin Core - are you still working on this issue? Please submit a WIP PR or comment back within the next 3 days or you will be removed from this ticket and it will be returned to an ‘Open’ status. Please let us know if you have questions!
- [x] warning (3 days)
- [ ] escalation to mods (6 days)
Funders only: Snooze warnings for 1 day | 3 days | 5 days | 10 days | 100 days
I've been working on a dockerfile to build KEVM. It's coming along. I also need to contact the K-Semantics group (@cdetrio said he'd connect me, so no holdups there as of now). Still moving along, just a longer project.
Just had a call with Everett from the K-Semantics group. We talked about what the best path forward will be and I think we have a good path.
What I'll be focusing on is integrating the KEVM into go-ethereum and creating a client in hive to be able to run the tests. I'll start by doing a fork to the go-ethereum client and setting up the client that way. If we want to have a PR to bring the KEVM into go-ethereum (so it can be run with a flag) we could also do that, but for the moment I'll just start on my fork in any case.
Once we have the client setup, it should be good to start reporting results like other clients.
I think making a PR into Geth which allows flagging that KEVM be the underlying execution engine is a good "victory point" for this PR, as everything after that is (1) fixing bugs we find in KEVM and (2) extending KEVM to do more of the work and leave less of the work to Geth.
I've asked @dwightguth about how KEVM is hooked up to Mantis, and he provided this link which describes the protobuf communication between the OCaml backend and Mantis: https://github.com/input-output-hk/mantis-extvm-pb/blob/master/msg.proto
He also says:
the protocol consists of sending a Hello message to the kevm vm followed by one or more CallContext messages, each of which represents a transaction to execute
the kevm vm will return a CallResult message
meanwhile they need to be able to respond via the same IPC (which is done using tcp currently) to GetAccount, GetStorageData, GetCode, and GetBlockhash messages
finally, once they receive the CallResult, they need to iterate through the various pieces of that message and apply the relevant updates to the state
note that this protobuf doesn't yet handle Byzantium or Constantinople so they will likely need to make pull requests to add that functionality as well
it will basically consist of modifying this file to add block number fields for both to the EthereumConfig message
and then we will have to make a minor change to the evm semantics to get it to read that data and select the correct schedule
there is also the question of code hashes; they will likely need to add a codeHash field to the Account message, and we will need to make some minor improvements to the evm semantics, if we want to support constantinople's EXTCODEHASH opcode
@somniastellarum Hello from Gitcoin Core - are you still working on this issue? Please submit a WIP PR or comment back within the next 3 days or you will be removed from this ticket and it will be returned to an ‘Open’ status. Please let us know if you have questions!
- [x] reminder (3 days)
- [ ] escalation to mods (6 days)
Funders only: Snooze warnings for 1 day | 3 days | 5 days | 10 days | 100 days
I've created my fork of the geth codebase and I've been trying to figure out how to do this integration. I was sort of hopeful (even though I knew it likely wasn't going to workout) that I could just create a type that satisfies the EVM interface (I think it's VM interface in the codebase). But I think it's slightly off for exactly how the KEVM interface will interact with it. Is there someone on the geth side that I can talk to about exactly how to do this integration? @cdetrio this might be a question for you. On the KEVM side, for the changes to update to Byzatium or Constantinople, did you want me to handle some of those or will that be through the core team? If I don't have to worry about those changes and focus instead on the geth side, it should be relatively straightforward. Similarly, did you want me to help with the msg.proto update? That should be relatively straightforward so I certainly don't mind doing that. For the moment, I'm concentrating more on the Geth side in any case, but I'll adapt as the final plan comes together.
@somniastellarum Hello from Gitcoin Core - are you still working on this issue? Please submit a WIP PR or comment back within the next 3 days or you will be removed from this ticket and it will be returned to an ‘Open’ status. Please let us know if you have questions!
- [x] reminder (3 days)
- [ ] escalation to mods (6 days)
Funders only: Snooze warnings for 1 day | 3 days | 5 days | 10 days | 100 days
@SomniaStellarum KEVM itself can handle Byzantium and Constantinople, the the msg.proto needs to be updated to pass the needed data for those back and forth I believe. @dwightguth correct me if I'm mistaken.
I should be able to get msg.proto modified on my end. If I run into problems I'll check back here to get some help. In the meantime I'm still working through the geth code to get this integrated. My thinking right now is to replace the EVM struct with an interface and implement that interface with KEVM (communicating through msg.proto). I'm still working through how complicated that will be. It depends if there are any parts of code that really rely on the EVM struct. I'll keep you posted.
@somniastellarum Hello from Gitcoin Core - are you still working on this issue? Please submit a WIP PR or comment back within the next 3 days or you will be removed from this ticket and it will be returned to an ‘Open’ status. Please let us know if you have questions!
- [x] reminder (3 days)
- [ ] escalation to mods (6 days)
Funders only: Snooze warnings for 1 day | 3 days | 5 days | 10 days | 100 days
@somniastellarum Hello from Gitcoin Core - are you still working on this issue? Please submit a WIP PR or comment back within the next 3 days or you will be removed from this ticket and it will be returned to an ‘Open’ status. Please let us know if you have questions!
- [x] reminder (3 days)
- [ ] escalation to mods (6 days)
Funders only: Snooze warnings for 1 day | 3 days | 5 days | 10 days | 100 days
There are a couple issues with what I was previously planning, so I think I'm somewhat changing course (on the go-ethereum side that is). Creating an interface for the EVM, while a good idea, has some problems. There is at least one place where on top of the methods that get called, the public fields are referenced. Bringing this into a method would be possible, but it's starting to make this a bigger job than just changing a couple references from the struct to the interface to being a bigger change than I initially thought. I do think it would be a good idea, but for now I think the path forward for this issue is to limit it's scope to just getting KEVM working with geth. As such, I've started what I think will be the path forward.
Essentially, it will be modifying the EVM definition so that when KEVM is requested it basically passes on the calls to it instead of using the standard EVM. I think the methods that it really needs to take care of are Call and Create. These both would send a CallMessage (after the Hello) to KEVM. I don't think we need to create a new goroutine to wait for the reponse and respond to any calls from KEVM, but it might be needed. Also, this might need a bit of tweaking in order to get the correct information to KEVM, but I think it's manageable.
The version of proto in vendor is older, so I replaced that for testing right now. I'm thinking that is generating the *.pb.go file during the build process. Right now I'm working of Windows, so I can't quite test on Linux, but I'll do that before completing my final commits. Can anyone confirm if that's how the pb.go gets generated? Also, would there be an objection to updating proto?
On the msg.proto side, on top of the changes that were already asked for above, we'll need to add a go package option as otherwise it uses a different package name that doesn't work too well for go.
Once this is all integrated, to test the integration with hive, I'm basically going to create a client that will point to my repo temporarily. It will call geth with a --UseKEVM flag (or something like that, still TBD) and if it behaves properly, I'll update all the PR's so that they are pointed to the right repo's and submit the PR's.
Let me know if anyone has any major objections to my path forward.
FYI: My repo https://github.com/SomniaStellarum/go-ethereum
and the initial commit (lots still to do) https://github.com/SomniaStellarum/go-ethereum/commit/bd70424a47eeac7a17a120334d07a7a0a5e5c1e8
@SomniaStellarum that sounds good. You may also look into EEI https://github.com/ewasm/design/blob/master/eth_interface.md if you have time, but it might not be worth it for this particular bounty. For the most part, KEVM can be made to follow the EEI. It's a separation of the EVM computational engine from the blockchain-level stuff, which should make VMs more "pluggable" than before.
But I think whatever you do to get it together will be fine, it can be improved later to be made to follow the EEI better, as there aren't many EEI-compliant clients yet anyway.
I have done a bit of work on a KEEI, https://github.com/kframework/eei-semantics
The idea would be that other tools could interface directly through the EEI, and then we can swap out the VM (so we can use KEVM, KIELE, or KWASM) under the hood. This is how the ewasm team is planning on plugging wasm VMs into Ethereum clients as well.
BTW, if you want another call to discuss the project, send me an email.
@somniastellarum Hello from Gitcoin Core - are you still working on this issue? Please submit a WIP PR or comment back within the next 3 days or you will be removed from this ticket and it will be returned to an ‘Open’ status. Please let us know if you have questions!
- [x] reminder (3 days)
- [ ] escalation to mods (6 days)
Funders only: Snooze warnings for 1 day | 3 days | 5 days | 10 days | 100 days
Awesome project! Thanks @gitcoinbot for your $1000 funding, and thanks @SomniaStellarum for taking the challenge. Let's double the fun: https://github.com/runtimeverification will award you a $1000 bonus at completion, @SomniaStellarum. Please stay in touch with @ehildenb and let us know when done.
@grosu Awesome, Thank you so much. I really appreciate the support. :)
@ehildenb EEI looks really good. I've actually seen some of the EWASM code as I've been doing this work, so I can see how it will have some overlap. For now, I'm going to be focusing integrating KEVM, but if that goes well, maybe I can connect with the EWASM group to doing some more general work like meeting the EEI standard. This would likely be where I'd start looking at doing some more Interface (the golang version here) work to pull out the EVM into it's own interface and allow KEVM or EWASM to satisfy that interface and be a bit more Plug and Play.
As I was going through setting up the protobuf stuff, I ran into some questions. I don't think we need a call for these, so I'll just ask them here. Though if you know any EWASM people to talk to, I might reach out to them sooner than later.
First questions is likely the most important. Earlier, you mentioned that one or more CallContext's (transactions) get sent to KEVM. My question is just about the timing of these messages. Is KEVM expecting more than one or if we're sending them one at a time, how will it behave? Right now, geth is really just sending them one at a time (or that's likely the easiest way to integrate it). Also, how does it handle recursive calls? I'm guessing it creates the recursive call and returns the results for all the calls, but can you confirm that?
Next question, based on msg.proto, I'm guessing KEVM generates the new contract address. What kind of checks does it do before running the call and returning the result? Does it check if there is already another contract at that address? Or does that need to be done on the geth side?
Finally, I did have a question about the serialization. In msg.Proto, there are a bunch of fields (blocknumbers, gas, addresses) that are passed around a bytes. What is the serialization of these fields? Are you expecting the underlying byte data or are these fields passed by text?
Let me know if you have any questions.
Hey @SomniaStellarum, I'll do my best to answer your questions, but some of the more specific ones about the interactions between an external client via the protobuf will need @dwightguth to chime in.
KEVM will handle recursive calls and creates itself, it has the semantics of these things already setup. The way the current hookup with Mantis looks like this (as far as I understand):
- Mantis sends a
CallContextto KEVM, which starts execution. - Every step, KEVM is checking "is there any global state I need from the client?" (eg. does it have the account information it needs for a
CALL/EXTCODECOPYit's trying to execute loaded?) - If there is state it needs, it sends a request for that data back to Mantis, and when it receives the reply with the data, it stores it in it's own local copy of the state (so that future queries to the same data will not trigger a call out to Mantis).
So I think you'll also need Geth setup to respond to the calls that KEVM makes back regarding various components of state.
KEVM isn't doing very much sanity checking on the data it gets back from the client though. It basically assumes that you are handing it valid information. It also does not do consensus work (will have to rely on client for that).