[WIP] EOF support

[harkal]

!!!This is WIP!!! !!! After rebasing to vyperlang/vyper master, tests fail, needs some fixes !!!

What I did

• New opcodes support RJUMP, RJUMPI, JUMPF, CALLF, RETF
• Enhanced the assembler to track function splits and produce the information required to emit proper EOFv1 bytecode
• Updated the opcode utility functions:
  • to support reverse mapping of opcodes to mnemonics
  • detect terminating opcodes
  • get the number of immediate values for every instruction
• Removed the support for JUMPF
• Removed the support for PC
• Support for creating a "bundle" of two consecutive EOFv1 bytecodes in a single container for deployment. While the commands for copying the code continue to operate on the whole container
• Made reject into a function call
• Removed the pushing of return addresses in the operand stack, as CALLF, and RETF operate on the new execution stack automatically
• Modified the opcode output so that it shows the functions and their properties (inputs, outputs, max stack height)
• Created a validation library and cli tool to validate EOFv1 bytecode, to be used for testing the compiler's output. This is currently always active on the compiler, but can be part of automated tests eventually. Validations that currently happens:
  • Headers and version
  • The existence or not of mandatory EOF sections
  • The order they come at
  • Limits on the number of code sections, types and data
  • The proper sequencing of the streaming headers, code and data and whether it all makes sense and finishes exactly where it should
  • Validates the inputs, outputs are in rang. Zero for the first code segment
  • Validates each code segment for valid instructions with proper number of immediates
  • Checks relative jumps and conditional jumps for out of bound destinations
  • Checks for invalid code section ids for CALLF
  • Checks is the opcodes don't go over section's end
  • Checks if last opcode of each code section is a terminating one
  • Checks if relative jumps don't end up in immediates
• Manually tested 'actual' execution of several small EOFv1 programs with the use of geth's evm
• Added the --experimental-eof command line argument that enables all of the new functionality and made sure everything else works like before without it
• Calculate the max stack height for every function

What is next

• Possibly separate the responsibilities for IR nodes label, goto, exit_to and possibly add a call and return node to make it for cleaner, more understandable code.
• Get an EVM that supports EOFv1 and integrate it, so we can start full testing. Finally, there are several implementations that support it, however not pyevm as far as I can tell. Plugging in a different one seems like a lot of work. Maybe bringing pyevm up to speed, especially now that there are other implementations out, is going to be more efficient?

[codecov-commenter]

Codecov Report

Attention: Patch coverage is 24.93703% with 298 lines in your changes missing coverage. Please review.

Project coverage is 86.96%. Comparing base (e9db8d9) to head (25b0cee).

:exclamation: Current head 25b0cee differs from pull request most recent head d9a33ae

Please upload reports for the commit d9a33ae to get more accurate results.

Files	Patch %	Lines
vyper/evm/eof.py	11.22%	166 Missing :warning:
vyper/ir/compile_ir.py	31.66%	71 Missing and 11 partials :warning:
vyper/compiler/output.py	41.66%	11 Missing and 3 partials :warning:
vyper/cli/validate_eof.py	0.00%	13 Missing :warning:
vyper/evm/opcodes.py	50.00%	12 Missing :warning:
vyper/compiler/phases.py	40.00%	4 Missing and 2 partials :warning:
vyper/codegen/self_call.py	50.00%	1 Missing and 2 partials :warning:
vyper/compiler/__init__.py	50.00%	1 Missing and 1 partial :warning:

Additional details and impacted files

@@            Coverage Diff             @@
##           master    #3457      +/-   ##
==========================================
- Coverage   91.37%   86.96%   -4.41%     
==========================================
  Files         109       86      -23     
  Lines       15606    11151    -4455     
  Branches     3432     2554     -878     
==========================================
- Hits        14260     9698    -4562     
- Misses        916     1036     +120     
+ Partials      430      417      -13     

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.

[charles-cooper]

i think the failures are coming from default evm version being eof, but we already can't test cancun since py-evm does not support yet (cf. notes in https://github.com/vyperlang/vyper/pull/3373)