FastAPI-template icon indicating copy to clipboard operation
FastAPI-template copied to clipboard

Published 20 hours ago • verified publisher icon s3rius

Worker process interrupted

Open rcholic opened this issue 1 year ago • 5 comments
trafficstars

I use taskiq with ListQueueBroker (backed by reddis). In the log, I see a lots of worker process interrupted:

2024-10-23 22:22:38.363 | INFO     | trace_id=0 | span_id=0 | logging:callHandlers:1706 - Listening started.
2024-10-23 22:22:39.445 | WARNING  | trace_id=0 | span_id=0 | logging:callHandlers:1706 - Worker process interrupted.
2024-10-23 22:22:39.446 | WARNING  | trace_id=0 | span_id=0 | logging:callHandlers:1706 - Shutting down the broker.
2024-10-23 22:22:42.697 | INFO     | trace_id=0 | span_id=0 | logging:callHandlers:1706 - Listening started.
2024-10-23 22:22:43.684 | WARNING  | trace_id=0 | span_id=0 | logging:callHandlers:1706 - Worker process interrupted.
2024-10-23 22:22:43.686 | WARNING  | trace_id=0 | span_id=0 | logging:callHandlers:1706 - Shutting down the broker.
2024-10-23 22:22:46.688 | INFO     | trace_id=0 | span_id=0 | logging:callHandlers:1706 - Listening started.
2024-10-23 22:22:47.921 | WARNING  | trace_id=0 | span_id=0 | logging:callHandlers:1706 - Worker process interrupted.
2024-10-23 22:22:47.923 | WARNING  | trace_id=0 | span_id=0 | logging:callHandlers:1706 - Shutting down the broker.
2024-10-23 22:22:50.809 | INFO     | trace_id=0 | span_id=0 | logging:callHandlers:1706 - Listening started.
2024-10-23 22:22:52.167 | WARNING  | trace_id=0 | span_id=0 | logging:callHandlers:1706 - Worker process interrupted.
2024-10-23 22:22:52.181 | WARNING  | trace_id=0 | span_id=0 | logging:callHandlers:1706 - Shutting down the broker.
2024-10-23 22:22:55.158 | INFO     | trace_id=0 | span_id=0 | logging:callHandlers:1706 - Listening started.
2024-10-23 22:22:56.489 | WARNING  | trace_id=0 | span_id=0 | logging:callHandlers:1706 - Worker process interrupted.
2024-10-23 22:22:56.492 | WARNING  | trace_id=0 | span_id=0 | logging:callHandlers:1706 - Shutting down the broker.

Is there a way to keep the worker and broker keep running?

rcholic avatar Oct 24 '24 02:10 rcholic

Can you enable DEBUG logging to see where the error might be?

s3rius avatar Oct 24 '24 07:10 s3rius

Can you enable DEBUG logging to see where the error might be?

hmm, I do have following setting turned on for debug mode in settings, but I don't see errors that lead to the worker interruption issue. Anywhere elese I can enable DEBUG logging? thanks!

log_level: LogLevel = LogLevel.DEBUG

rcholic avatar Oct 24 '24 14:10 rcholic

Can you supply taskiq worker cli with --log-level DEBUG argument?

Like taskiq worker --log-level DEBUG ....

s3rius avatar Oct 24 '24 16:10 s3rius

Can you supply taskiq worker cli with --log-level DEBUG argument?

Like taskiq worker --log-level DEBUG ....

I'm using docker-compose to start the worker like following:

  taskiq-worker:
    <<: *main_app
    labels: []
    command:
    - taskiq
    - worker
    - -fsd
    - my_app.tkq:broker

to add log-level , is following correct?

  taskiq-worker:
    <<: *main_app
    labels: []
    command:
    - taskiq
    - worker
    - -fsd
    - --log-level DEBUG
    - my_app.tkq:broker

rcholic avatar Oct 25 '24 03:10 rcholic

Yes, should be correct.

s3rius avatar Oct 25 '24 08:10 s3rius