WindowsIteration terminating abnormally due to MemoryError

[antnks]

Fuzzing process randomly dies with the following Python exception:

2019-02-01 09:13:19,769 INFO certfuzz.minimizer.minimizer_base - start=79 min=79 target_guess=1 curr=39 chance=0.50000 miss=0/11 total_misses=0/1 u_crashes=0
2019-02-01 09:13:38,473 INFO certfuzz.minimizer.minimizer_base - testcase=0x3c7553b8.0xd8e24aea signal=None
2019-02-01 09:13:45,084 INFO certfuzz.minimizer.minimizer_base - start=79 min=39 target_guess=1 curr=21 chance=0.50000 miss=0/11 total_misses=0/2 u_crashes=1
2019-02-01 09:14:11,051 WARNING certfuzz.iteration.iteration_windows - WindowsIteration terminating abnormally due to MemoryError:
Traceback (most recent call last):
  File "bff.py", line 51, in <module>
    main()
  File "C:\BFF\certfuzz\bff\windows.py", line 22, in main
    bff()
  File "C:\BFF\certfuzz\bff\common.py", line 131, in go
    campaign.go()
  File "C:\BFF\certfuzz\campaign\campaign_base.py", line 537, in go
    self._do_interval()
  File "C:\BFF\certfuzz\campaign\campaign_base.py", line 512, in _do_interval
    self._do_iteration(sf, r, seednum)
  File "C:\BFF\certfuzz\campaign\campaign_windows.py", line 186, in _do_iteration
    iteration()
  File "C:\BFF\certfuzz\iteration\iteration_base.py", line 269, in go
    self.process_testcases()
  File "C:\BFF\certfuzz\iteration\iteration_base.py", line 260, in process_testcases
    pipeline.go()
  File "C:\BFF\certfuzz\tc_pipeline\tc_pipeline_base.py", line 299, in go
    self.analysis_pipeline.send(testcase)
  File "C:\BFF\certfuzz\tc_pipeline\tc_pipeline_base.py", line 121, in verify
    target.send(testcase)
  File "C:\BFF\certfuzz\tc_pipeline\tc_pipeline_base.py", line 131, in minimize
    self._minimize(testcase)
  File "C:\BFF\certfuzz\tc_pipeline\tc_pipeline_base.py", line 227, in _minimize
    m.go()
  File "C:\BFF\certfuzz\minimizer\minimizer_base.py", line 723, in go
    self.swap_bytes()
  File "C:\BFF\certfuzz\minimizer\minimizer_base.py", line 881, in swap_bytes
    self.seed, self.fuzzed_content)
  File "C:\BFF\certfuzz\minimizer\minimizer_base.py", line 907, in bytewise_swap2
    append(a)
MemoryError

What would you suggest to troubleshoot?

[antnks]

This is how memory usage looks like just before the termination:

[wdormann]

I've not been able to track down the cause of this problem. Mostly because it seems to be nondeterministic. In particular, in my testing, BFF's memory usage isn't a continual growth where you can see what is being leaked. But rather there are occasional times where the usage grows to the point where it's too much for a 32-bit Windows python process to handle.

If you run BFF in debug mode, you can use heapy to see how memory is being used. Maybe you'll have better luck than I have had?

See also #15. In my testing using 64-bit python seems to be a viable workaround.

[antnks]

The issue is indeed memory related, as well as related to a non-optimized array handling I tried to mitigate the crash by adding try/catch and force minimizer end: https://github.com/CERTCC/certfuzz/pull/29 With this change I did not get crashes after 4 days of running, but there are still performance issues when fuzzing big files using a 32 bit Python

[wdormann]

Thanks! I'll have a look and incorporate the change if it passes the smoke test.