bullmq icon indicating copy to clipboard operation
bullmq copied to clipboard

Flows: job.getChildrenValues is not a function for sandboxed workers

Open bpinney opened this issue 4 years ago • 8 comments

Version 1.46.6 The below line works as documented for normal workers, but throws the error job.getChildrenValues is not a function for sandboxed processors.

const childrenValues = await job.getChildrenValues();

Below are code snippets used to test, sans imports for brevity. A nested job added to a FlowProducer:

await flowProducer.add({
    name: "parent-job",
    queueName: "parent",
    children: [{
      name: "child",
      queueName: "child",
      data: { colour: "red" },
    }]
  });

Processed by sandboxed worker processes:

const parentWorker = new Worker(
  "parent", 
  path.join(__dirname, "../dist") + "/processor-parent.js",
);

const childWorker = new Worker(
  "child", 
  path.join(__dirname, "../dist") + "/processor-child.js",
);

Processor definitions:

import { processorParent } from "./processors";
export default processorParent;

import { processorChild } from "./processors";
export default processorChild;

processors.ts:

export const processorParent = async(job: Job) => {
  const childrenValues = await job.getChildrenValues();
  console.log(childrenValues);
}

export const processorChild = async (job: Job) => {
  return "Child Return Test Value"
}

bpinney avatar Sep 15 '21 20:09 bpinney

Yes, we are missing some proxies for flow related methods in Flows. Thanks for reporting.

manast avatar Sep 16 '21 01:09 manast

Current workout:

import { Job, Queue } from "bullmq"

export default async function(sandboxedJob) {
    const queue = new Queue(sandboxedJob.queueName, { connection }) // pass the same connection settings
    const job = await Job.fromId(queue, sandboxedJob.id)

    const childrenValues = await job.getChildrenValues()
}

AlexOwl avatar Aug 15 '22 19:08 AlexOwl

Any updates on this?

fmoessle avatar Nov 19 '22 12:11 fmoessle

I am also looking for this to be resolved.

imperfect-circuits avatar Mar 14 '23 07:03 imperfect-circuits

+1

oxypomme avatar Sep 21 '23 14:09 oxypomme

An updates on this?

shaunakv1 avatar Jul 12 '24 14:07 shaunakv1

I just got bit by this thinking it was supported. Rewrote a large chunk of my app and ran into this issue. I should've checked it first with a small example 😞.

tolgap avatar Aug 24 '24 13:08 tolgap

We have a PR for this, seems like it could be merged anytime now https://github.com/taskforcesh/bullmq/pull/1417

manast avatar Aug 26 '24 09:08 manast

An update on this?

alimoezzi avatar Oct 15 '24 13:10 alimoezzi

issue still here on v5.34.5

younes101020 avatar Jan 08 '25 12:01 younes101020

Added in v5.35.0: https://docs.bullmq.io/changelog#id-5.35.0-2025-01-22

manast avatar Jan 22 '25 09:01 manast

@manast -- just wondering your thoughts on making the timeout adjustable when getting children values. We have some large flows with many child jobs with some fairly large return values.

While we haven't hit that timeout yet, just wondering if that ceiling could be adjustable if needed.

rosslavery avatar Jan 22 '25 21:01 rosslavery

The easiest way would be to configure it via env variables.

manast avatar Jan 23 '25 10:01 manast

If the call takes too long time thats not good in general. We may need to add pagination support to keep the calls very fast, as keeping Redis busy with one command for so long is not recommended.

manast avatar Jan 23 '25 10:01 manast

I'm on latest bullmq (5.63.0), SandboxJob doest have getChildrenValues,

Image

Is that expected?

felixmosh avatar Nov 04 '25 20:11 felixmosh