fuzzbench
fuzzbench copied to clipboard
google
Presubmit failure: AssertionErrors
Hi, any ideas on how to fix the following assertion errors? (I get them when I run make presubmit):
=================================== FAILURES ===================================
____________________ test_benchmark_rank_by_stat_test_wins _____________________
... [make_presubmit_log.txt](https://github.com/google/fuzzbench/files/6251987/make_presubmit_log.txt)
> assert ranking.equals(expected_ranking)
E assert False
E + where False = <bound method NDFrame.equals of fuzzer\nafl 0\nlibfuzzer 0\nName: stat wins, dtype: int64>(libfuzzer 0\nafl 0\ndtype: int64)
E + where <bound method NDFrame.equals of fuzzer\nafl 0\nlibfuzzer 0\nName: stat wins, dtype: int64> = fuzzer\nafl 0\nlibfuzzer 0\nName: stat wins, dtype: int64.equals
analysis/test_data_utils.py:296: AssertionError
______________________ test_experiment_rank_by_num_firsts ______________________
`def test_experiment_rank_by_num_firsts():
experiment_df = create_experiment_data()
snapshots_df = data_utils.get_experiment_snapshots(experiment_df)
ranking = data_utils.experiment_level_ranking(
snapshots_df, data_utils.benchmark_rank_by_median,
data_utils.experiment_rank_by_num_firsts)`
`expected_ranking = pd.Series(index=['libfuzzer', 'afl'], data=[1.0, 1.0])`
> assert ranking.equals(expected_ranking)
E assert False
E + where False = <bound method NDFrame.equals of fuzzer\nafl 1.0\nlibfuzzer 1.0\nName: number of wins, dtype: float64>(libfuzzer 1.0\nafl 1.0\ndtype: float64)
E + where <bound method NDFrame.equals of fuzzer\nafl 1.0\nlibfuzzer 1.0\nName: number of wins, dtype: float64> = fuzzer\nafl 1.0\nlibfuzzer 1.0\nName: number of wins, dtype: float64.equals
analysis/test_data_utils.py:339: AssertionError
__________________________ test_fuzz_function_errors ___________________________
`def test_fuzz_function_errors():
"""This test calls fuzzer_module.fuzz() under circumstances in
which it should throw an exception. If the call exceptions, the
test passes, otherwise the test fails. This ensures that we can
properly detect failures during fuzzing."""
for fuzzer_module in _get_all_fuzzer_modules():
with pytest.raises(Exception) as error, Patcher():
fuzzer_module.fuzz('/input-corpus', '/output-corpus',
'/target-binary')`
# Type error probably means module is doing something else wrong,
# so fail if we see one. If that is not the case than this assert
# should be removed.
> assert not isinstance(error.value, TypeError)
E assert not True E + where True = isinstance(TypeError("read() missing 1 required positional argument: 'numBytes'"), TypeError) E + where TypeError("read() missing 1 required positional argument: 'numBytes'") = <ExceptionInfo TypeError("read() missing 1 required positional argument: 'numBytes'") tblen=4>.value
fuzzers/test_fuzzers.py:81: AssertionError
FuzzBench only support <= Python 3.8, which version of python are you using ?
I am using Python 3.8.8
On Sat, Apr 3, 2021, 2:38 AM Abhishek Arya @.***> wrote:
FuzzBench only support <= Python 3.8, which version of python are you using ?
— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/google/fuzzbench/issues/1125#issuecomment-812828524, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABKCR5KMPMFEFPXYION2QUDTG3AXPANCNFSM42J3EF4A .
Cannot reproduce and have no clue on this. @lszekeres since he wrote this code, maybe he has an idea to debug this.