Tests in Py-EVM are much slower than in PyEthereum

[sdaveas]

• py-evm Version: 0.3.0a9
• OS: linux 5.4.3-arch1-1
• Python Version: 3.8.0
• Environment:

attrdict==2.0.1
attrs==19.3.0
base58==1.0.3
blake2b-py==0.1.3
cached-property==1.5.1
certifi==2019.11.28
chardet==3.0.4
cytoolz==0.10.1
eth-abi==2.0.0
eth-account==0.4.0
eth-bloom==1.0.3
eth-hash==0.2.0
eth-keyfile==0.5.1
eth-keys==0.2.4
eth-rlp==0.1.2
eth-tester==0.3.0b1
eth-typing==2.2.1
eth-utils==1.8.1
hexbytes==0.2.0
idna==2.8
ipfshttpclient==0.4.12
jsonschema==3.2.0
lru-dict==1.1.6
multiaddr==0.0.8
mypy-extensions==0.4.3
netaddr==0.7.19
parsimonious==0.8.1
protobuf==3.11.1
py-ecc==1.7.1
py-evm==0.3.0a9
py-solc==3.2.0
py-solc-x==0.6.0
pycryptodome==3.9.4
pyethash==0.1.27
pyrsistent==0.15.6
python-bitcoinlib==0.10.2
requests==2.22.0
requests-cache==0.5.2
rlp==1.2.0
semantic-version==2.8.3
six==1.13.0
solc==0.0.4
toolz==0.10.0
trie==1.4.0
urllib3==1.25.7
varint==1.0.2
web3==5.3.0
websockets==8.1

What is wrong?

I am running a piece of Solidity code at PyEthereum and Py-EVM and I observed performance degradation in the case of Py-EVM.

Here are two representative tests that outline the performance issue:

Test 1 - Storage

function test_1() 
public 
returns(bool) 
{
    uint[VAR] memory buffer;
    buffer[0] = 1;
    return buffer[0] == 1;
}

VAR: 10^5

Environment	Time (sec)
PyEthereum	1.498
Py-EVM	10.014

VAR: 10^6

Environment	Time (sec)
PyEthereum	3.901
Py-EVM	47.47

---

Test 2 - Iteration

Code

function benchmark() 
public 
returns(bool) 
{
    uint sum=0;
    for (uint i=0; i<VAR; i++) {
        sum+=i;
    }
    return sum < 1<<15;
}

VAR: 10^4

Environment	Time (sec)
PyEthereum	1.448
Py-EVM	14.642

VAR: 10^5

Environment	Time (sec)
PyEthereum	2.749
Py-EVM	144.88

---

The complete code of the tests along with the flamegraphs of the above examples can be found . I am aware of #1213 but given that this issue was opened about a year ago, I am wondering if I am losing any improvements?

[pipermerriam]

I'll admit I'm surprised at the degree to which the two differ, but not surprised Py-EVM is slower.

The modular design of the library incurs non-trivial overhead due to each opcode execution resulting in a new call frame being created. This is an area that I suspect we'll make some progress on this year once Trinity goes into beta and we can start focusing on finer tuning as opposed to just getting the core functionality in place.

If you happen to identify any areas where there are obvious optimizations available we'd be more than happy to look into them.

[sdaveas]

Thanks! I will let you know if anything comes up.

[gtklocker]

Kinda offtopic: I'm understanding that running computation heavy code is probably not the best use-case for this library. Is there an alternative to the old PyEthereum which is in-memory and fast that can be used nowadays?