node-worker-threads-pool icon indicating copy to clipboard operation
node-worker-threads-pool copied to clipboard

Can't find class at runtime

Open appocalypseltd opened this issue 2 years ago • 2 comments

I may be demonstrating my deep, deep ignorance of threading in node, but this is my first attempt at using worker threads, and the documentation says I ought to use a worker pool.

So here we are. I am using the node-worker-threads-pool package. My app will be training several dozen ML models simultaneously using TensorFlow.

The following TypeScript code falls over at runtime, although it transpiles fine:

trainModels = async () => {
   const modelIds: string[] = getModelId();

   for await (const modelId of modelIds) {
      this.dynamicPool.exec({
         task: id => {
            const predictor = new Predictor(id, []);
            // ...do such-and-such
         },
         param: modelId,
      });
   }
};

The runtime complains:

ReferenceError [Error]: Predictor is not defined

So it can't find it. Accessing the class elsewhere in the code is fine; but not within a thread.

I"m guessing this is threading 101 in node. How do I get around this? Perhaps I can construct the class and then create the thread within that class instead? What is the pattern?

appocalypseltd avatar Feb 04 '23 21:02 appocalypseltd

Ok, so I've done this. Seems to be happier so far:

constructor(private pairNames: string[], private data: ModelData[]) {
   for (const pairName of pairNames) {
      this.dynamicPool.exec({
         task: this.execTask,
         param: pairName,
      });
   }
   console.log(`In Predictor constructor`);
}

private execTask = async (pairName: string) => {
   console.log(`In Predictor execTask for ${pairName}`);
};

appocalypseltd avatar Feb 04 '23 21:02 appocalypseltd

The document says that you cannot access variables outside the task function. So that means we cannot have context in the task function, like closure does.

task Function as a task to do. ⚠️Notice: You can NOT access variables defined outside the task function!

hhu8 avatar Sep 07 '23 19:09 hhu8