tatoeba2 icon indicating copy to clipboard operation
tatoeba2 copied to clipboard

Increase Queue maxworkers to 2

Open trang opened this issue 2 years ago • 7 comments

CK reported that it sometimes took several minutes (up to 19 minutes) to import audio (see https://github.com/Tatoeba/tatoeba2/issues/2955#issuecomment-1163012139).

I noticed that when we run a worker, even if the runworker command is executed at 0 second, the created date is at 1 second and sometimes at 2 seconds.

Example:

MariaDB [tatoeba]> select * from queue_processes;
+-----+-------+---------------------+---------------------+-----------+--------+------------------------------------------+
| id  | pid   | created             | modified            | terminate | server | workerkey                                |
+-----+-------+---------------------+---------------------+-----------+--------+------------------------------------------+
| 199 | 18943 | 2022-06-26 17:00:01 | 2022-06-26 17:01:22 |         0 | sloth  | 5b689d57e599b30b2549d4361488a07592877172 |
+-----+-------+---------------------+---------------------+-----------+--------+------------------------------------------+

Because of this, when the worker is terminated (after 20 minutes in our case) and we try to run the next worker, it fails because the previous worker is still running and we end up with no workers at all.

As a result, I have updated the config on prod to increase the maximum number of workers to 2. Not sure if there's a better solution.

trang avatar Jun 26 '22 17:06 trang