Graders should not consume grading requests when they are fully busy

[fushar]

Consider the following existing implementation:

https://github.com/ia-toki/judgels/blob/e2e4dfb6e1329aaed3bf5bc3f954af9055078b13/judgels-backends/judgels-grader-app/src/main/java/judgels/gabriel/grading/GradingRequestPoller.java#L36-L67

In this logic, each grader machine will consume a grading request message, then wait for up to one second, then consume another message, and so on.

This logic has a flaw: when we're regrading hundreds/thousands of submissions, especially with problems with high time limit, this can result in all grading requests to be prematurely consumed by the existing graders, even though individual worker threads are still busy grading initial submissions. This means that we cannot add more graders to help with the regrading, as all grading request messages have been consumed by the existing graders

The ideal situation is: a grader machine should not consume any grading request message when all the allocated worker threads are still busy.