fishtest
fishtest copied to clipboard
glinscott
Auto blocking workers?
I wonder if there is any appetite for auto blocking workers under some conditions. For example after two (three?) consecutive crash_or_time events.
I don't think that will be too ideal. The patch tested can be the culprit of both time losses and crashes.
That's a good point which I had not thought of. So the crashes/time losses should be at least for different runs and probably even for different users (in case a user submits a whole series of faulty patches at the same time).
So I would go for three consecutive crash_or_time events from three different runs from at least two different users.
That might work. you could probably check the residual of the run itself also, i.e. if this worker is the only one who crashes/time loses on the same test or something like that.
Take this as an example: https://tests.stockfishchess.org/tests/view/64f6425475cd1362d763aeed The patch is clearly bugged so all workers crash as opposed to only one.. so probably this could make it easier to catch somehow...