vdbergh

Assuming this can be made to work, then we still have a DOS situation when the proxy is down (which is bound to happen sometimes). So maybe, when `--fleet` is...

It is a DOS issue. The worker requests a task, which allocates resources on the server, and then dies. If this happens at high frequency the server will run out...

PS. I am not saying that this is the root cause why Fishtest is down right now, but it may have contributed to it.

@noobpwnftw I made a PR #1194 for the worker to use ghproxy. But it does not work. Can you have a look to see if I am using the proxy...

Well we also keep tasks even if they fail since this is useful for debugging. But I think event log entries take more space.

Deleted a comment since there wasn't something I hadn't thought of.

Trying again with the opposite conclusion. The recent flooding is really strange. If the github api call `rate_limit` fails then the server code will assign the worker a task from...

I just tested this theory by causing an exception after the call to `rate_limit` ```python def get_rate(): try: rate = requests.get( "https://api.github.com/rate_limit", timeout=HTTP_TIMEOUT ).json()["resources"]["core"] raise Exception("Github failure") #

Good point! It does not have to be an all or nothing situation.

Here is another possible reason for the flooding. `failed_task()` invokes `del_tasks()` whose implementation is this. ```python def del_tasks(run): if "tasks" in run: run = copy.deepcopy(run) del run["tasks"] return run ```...