echidna icon indicating copy to clipboard operation
echidna copied to clipboard

[Bug-Candidate]: Echidna multicore produces slower shrinking behavior

Open aviggiano opened this issue 2 years ago • 3 comments

Describe the issue:

I am running echidna with 8 workers, but I am getting unexpected/unwanted results for the shrinking step when a bug is found:

Instead of shrinking faster, when I add more workers the process is significantly slower.

Sometimes I see the shrink step go down, and then up, then down, and then up. It seems like increasing the number of workers adds a bunch of (duplicate) transactions that do not contribute to the smallest sequence possible.

I did a small 1-min test, reutilizing the corpus, with 1 worker vs 8 workers. You can see below that using more workers makes echidna run much slower when a bug is found.

1 worker

Screenshot 2023-08-16 at 15 52 02

8 workers

Screenshot 2023-08-16 at 15 49 21

Code example to reproduce the issue:

N/A

Version:

Echidna 2.2.1 slither 0.9.2

Relevant log output:

N/A

aviggiano avatar Aug 16 '23 18:08 aviggiano

I often encounter the same thing. I think this is because the different workers can overwrite each others "best shrunk" version and kind of work against each other that way.

rappie avatar Aug 19 '23 15:08 rappie

Yes, this is a known issue to be fixed

arcz avatar Aug 21 '23 09:08 arcz

I think minimization should be restricted to a single worker

gustavo-grieco avatar Sep 01 '23 12:09 gustavo-grieco