solid_queue icon indicating copy to clipboard operation
solid_queue copied to clipboard

Estimated number of threads is only valid for workers

Open ThomasCrambert opened this issue 11 months ago • 3 comments

Hello 👋,

The check estimated_number_of_threads assumes a worker will always be started. In case the configuration specifies to boot only a scheduler or a dispatcher, the estimation will return three (main + heartbeat + threads for each worker) instead of two (heartbeat + dispatcher or heartbeat + scheduler).

Maybe the problems comes from the default value for thread_count, it is 1 when it should be 0 if no workers are specified in the configuration, WDYT?

ThomasCrambert avatar Jan 20 '25 10:01 ThomasCrambert

I think the supervisor might need 3 connections as well: one for the main thread, one for the process maintenance task and one for its own heartbeat. The dispatcher, with default configuration, too: one for the polling, one for concurrency maintenance, and one for its heartbeat. I think 3 is a safe assumption so that you don't run out of DB connections, which is the purpose of this check.

rosa avatar Jan 20 '25 10:01 rosa

I think 3 is a safe assumption so that you don't run out of DB connections, which is the purpose of this check.

Fair enough 👍, this function might need a rewrite tho, when we looked into it we felt it was too focused on workers.

On an other note, does your explanation also apply to the scheduler? I'm unsure it would need at least three (main + heartbeat + schedule_recurring_tasks) or if it depends on the number of recurring tasks to process at once (so main + heartbeat + n * schedule_recurring_tasks)

ThomasCrambert avatar Jan 20 '25 10:01 ThomasCrambert

No, the scheduler should have enough with 2 (heartbeat + task scheduling).

rosa avatar Jan 20 '25 10:01 rosa

Going to close this one as the estimation and error shown should be ok regardless of whether you configure workers or not.

rosa avatar Jun 16 '25 20:06 rosa