syzkaller icon indicating copy to clipboard operation
syzkaller copied to clipboard

dashboard/app: capture and report probability of reproducing a bug

Open dvyukov opened this issue 6 years ago • 6 comments

Some repros trigger the bug immediately, some after some time, or not in each run. Several users asked for some kind of "how hard it is to re-trigger the crash using this repro" thing. E.g. what's average runtime? Multi-threaded/process repros are also some kind of signal.

dvyukov avatar Dec 18 '18 14:12 dvyukov

Also, how much percents of Multi-threaded bugs can reproduce by syzbot by now?

daydayup40 avatar Jan 27 '19 12:01 daydayup40

@daydayup40 I don't have such info. To get this number, one needs to go over all syzbot reported bugs and mark them as race/norace (https://syzkaller.appspot.com, https://syzkaller.appspot.com?fixed=upstream).

dvyukov avatar Jan 28 '19 08:01 dvyukov

@dvyukov Ok. thanks!

daydayup40 avatar Jan 28 '19 09:01 daydayup40

Besides threaded/non-threaded, it can also make sense to run it multiple times and observe how many times we reproduced it. We could also capture set of all crashes that the repro produces. But then we need to figure out where/how to present this. This info may also be useful during bisection. We currently use 10 VMs. We could use less for reliable bugs and more for unreliable bugs.

dvyukov avatar Apr 05 '19 12:04 dvyukov

How do you guyz calculate the probability ?

irenge avatar Oct 06 '21 14:10 irenge

How do you guyz calculate the probability ?

We don't calculate it atm.

dvyukov avatar Oct 07 '21 09:10 dvyukov