prom-client icon indicating copy to clipboard operation
prom-client copied to clipboard

No handling for non-cluster worker threads?

Open pies opened this issue 6 months ago • 4 comments

Hi,

Is there a way to get multi-threaded workers to report metrics? Thanks!

I.e.

import {spawn, Worker} from 'threads';

const worker = await spawn(
      new Worker('worker-code', params as any),
      {timeout: 40 * MINUTE_IN_MS},
) as ModuleThread<any>;

pies avatar Jun 16 '25 20:06 pies

After I finish #688 I might have time to look at this.

jdmarshall avatar Jul 11 '25 14:07 jdmarshall

I've been working in this code recently.

I have a PoC but I may be stuck.

I've been chatting with @zbjornson on another PR about making sure the aggregator doesn't gum up the primary thread when run in cluster mode, and it's dawned on me that rather than splitting up clusterMetrics() with setImmediates, it would be much simpler if the main thread could offload stats monitoring to a worker, which can then use all the CPU it wants.

Problem is the Node folks didn't get the worker API right on the first or second try (see also piscina, and https://nodejs.org/api/worker_threads.html#workerpostmessagetothreadthreadid-value-transferlist-timeout which is unstable but allows you to message arbitrary workers)

jdmarshall avatar Jul 13 '25 17:07 jdmarshall

I have a working prototype now, by using n + 1 BroadcastChannels

But it supports autodiscovery, and shunting collection into a Worker.

jdmarshall avatar Jul 14 '25 02:07 jdmarshall

#693

jdmarshall avatar Dec 09 '25 07:12 jdmarshall