Result queue clears slowly

[apapsch]

With save_limit set to 0, the result queue is clearing ever more slowly with each batch of tasks added. Setting a fixed limit for save_limit is not feasible in my case (and would still be high enough to trigger the slowing down).

My test case adds around 1000 tasks (which process very quickly), then waits for the results using result_group.

With an empty tasks table, the result queue is emptying in a somewhat acceptable pace. However, the delay between the execution of the last task and the return of result_group is already long and noticeable (many seconds). Repeating this a few times, the delay keeps increasing.

A real case will involve tens of thousands of tasks and the delay will be in the sky.

Do you have a hunch where the issue might hide, where I might look further?

[Grayknife]

@apapsch Can you test with Django Q2 version 1.6.2 to check if this behavior also occurs?

[apapsch]

I can confirm the issue does not occur with 1.6.2, or at least is not as pronounced. The delay starts at ca. 20 seconds, and in subsequent runs increases only slightly to 20 - 27 seconds.

Both 1.7.4 and 1.7.3 are affected, where the delay starts at 30 seconds and increases around 30 seconds each run.

I was not able to test 1.7.2, as it hit a bug, related to prometheus if I saw correctly.

[Grayknife]

Uhm it may be somehow related to this discussion i opened a few weeks ago https://github.com/django-q2/django-q2/discussions/241 .