scilla
scilla copied to clipboard
Zilliqa
Incorporate the dump of all deployed contracts into tests
There are around 1000 more or less unique contracts deployed on the Zilliqa blockchain as of time of creating this issue. We should incorporate these into our test suite in some form.
Some thoughts on this issue:
- it seems valuable to manually analyse several tens of most active (popular) contracts and make more thorough tests out of those;
- it would be great to record contract states and transitions and put them into the
scilla-runnertests, ensuring we can replicate real-world contracts' behaviors when improving or fixing Scilla.
I'm a little worried about possible copyright issues when storing these contracts in our test suite. And it will also significantly increase the size of the repository (so CI will be slower and it may be inconvenient for users to clone the repository).
I think another use case for such a test is to detect false positives/false negatives of a static analyzer.
This could be done as a separate script, that could be run manually (or via CI), which shows, how the static analyzer warnings on a contract base are changed between two commits. We could use it before releasing a new version of Scilla or create some integration with CI, that prints changes in warnings between master and the PR branch.
What do you think?
@jubnzv - I agree. We definitely oughtn't to store these in the test suite - it'll be enormous - but a tool to extract transitions from persistence and run the test might be useful (if we want reference data, we can store this hashed to avoid doing any copying).
Agreed re static analyzer - worth keeping in mind for when that is a possibility.
Seem fair?
Just to give more context on this:
- There are 970 unique Scilla contracts in the contract dump
- It's 428743 lines of code in total (average contract size is about 440 LoC)
- it's 15Mb of text
A lot of contracts are very similar with a diff of only around 10 lines or so.
