bullmq icon indicating copy to clipboard operation
bullmq copied to clipboard

[Bug]: Date transformed into string when queued

Open Holbanner opened this issue 1 year ago • 3 comments

Version

v5.8.2

Platform

NodeJS

What happened?

When adding an object with a key typed as a date in following format : 2024-09-13T15:00:00.000Z the worker gets it back as a string, logicaly loosing all date methods

How to reproduce.

Add an item with at least one js date key to the queue, then check the date in a worker. It is now a string

Relevant log output

No response

Code of Conduct

  • [X] I agree to follow this project's Code of Conduct

Holbanner avatar Sep 16 '24 07:09 Holbanner

Yeah, this is the way JSON.stringify/parse works. We would need another serialisation scheme to support dates.

manast avatar Sep 16 '24 07:09 manast

So the "proper" way to deal with this would be to rebuild the dates in the workers?

Holbanner avatar Sep 16 '24 08:09 Holbanner

Yes, you need to recreate the date objects when you need them.

manast avatar Sep 16 '24 08:09 manast

We are facing similar problems using date objects in our jobs. A nice solution could be to use superjson (https://github.com/flightcontrolhq/superjson) (or similar) to serialize and deserialize job data. This would automatically handle date objects and also bigint (https://github.com/taskforcesh/bullmq/issues/447) or map. I think it should not be very difficult to implement. For us, as we use a lot of workers, it would be helpful to have a global option to always use superjson for all jobs.

bredar avatar Oct 08 '24 15:10 bredar

We keep running into this issue again and again, because it's so counter-intuitive in a TypeScript codebase where people actually take types seriously.

Consider this (taken more or less directly from the docs:

type MyData = {
  someDate: Date
}

const worker = new Worker<MyData, void>(queueName, async (job: Job) => {
  console.log(typeof job.someDate)
  console.log(job.someDate.getFullYear())
})

(I'm hoping I'm reproducing this correctly with simplified basic code.)

typeof job.someDate will be string here, but tsc (and thus also IntelliSense etc.) will incorrectly mark it as Date. In other words: It will compile, but then fail at runtime, because getFullYear() isn't defined on a string. This, to me, goes against the very core of why we're using TypeScript: Having type-related issues at runtime is exactly what we don't want to have.

To resolve this, I can think of multiple ways:

Plain TypeScript

A TypeScript-native way would be to somehow restrict types for job data to only the ones that serialize/deserialize predictably. There could even be a config setting where this could be turned off and/or field types could be configured.

I haven't tested this, but I could imagine something like this (note: generated by Claude Sonnet 4.5):

// First, define what types are actually JSON-serializable
type JsonPrimitive = string | number | boolean | null;

type JsonSerializable = 
  | JsonPrimitive
  | JsonSerializable[]
  | { [key: string]: JsonSerializable };

// Now modify the Worker constructor to enforce this
class Worker<TData extends JsonSerializable, TReturn extends JsonSerializable> {
  constructor(
    queueName: string,
    processor: (job: Job<TData, TReturn>) => Promise<TReturn>
  ) {
    // implementation
  }
}

interface Job<TData = any, TReturn = any> {
  data: TData;
  // other properties...
}

The restriction on TReturn probably isn't necessary. And also, neither seems to account for void. But as I said: It's been generated by Claude.

More powerful JSON library

As was pointed out by @bredar, something like superjson could be used. While not perfect, it could at least fix some of the most common cases (like Date, bigint, Set and Map).

Fully-fledged schema library

Adopting something like zod could also be a good choice, since it provides great and easy ways to encode/decode custom types with proper type inference.

(The main reason why I'd pick zod over superjson is its adherence to standard-schema, which IMO makes it more likely to be used beyond just BullMQ.)


If you're open for one of these approaches, I'm happy to give it a try.

clemens avatar Nov 19 '25 14:11 clemens