[Bug] Throwing an exception from a workflow may result in `Failed to activate workflow` due to `DataCloneError`

[mjameswh]

Describe the bug

Throwing a non-Error and non-Failure exception and letting that exception bubble up out of the Workflow function causes activation failure (due to DataCloneError) if that object contains non-serializable objects.

Even though a Failed to activate workflow error message mentioning DataCloneError is visible in the log, the original exception is not accessible, making it difficult to figure out the origin of the failure.

We need to figure out how this situation can be handled better.

For example:

• We could consider serializing sink arguments immediately at the call site, rather than delaying that until completion of the activation, which would allow reporting the DataCloneError error in context (and also let's make that error message more obvious).
• By using a custom Serializer object, it might be possible to intercept serialization errors, and automatically replace problematic object by some safe alternative (ie. a JSON representation of that object, a string saying that item could not be serialized, etc).

Minimal Reproduction

1. Execute the following workflow

class MyCustomError {
  public readonly x: any;
  constructor(public message: string) {
    this.x = () => 1;
  }
}

export async function exampleWorkflow(): Promise<void> {
  throw new MyCustomError('Intentional error');
}

2. Observe that execution produces the following error:

2023-12-20T23:39:49.741Z [ERROR] Failed to activate workflow {
  namespace: 'default',
  taskQueue: 'hello-world',
  workflowId: 'workflow-X0CYgbSxElPenLeMFU68e',
  runId: '3fe2dbe4-8f71-4755-9bee-b6fc4d02cac4',
  workflowType: 'example',
  error: DataCloneError: ()=>1 could not be cloned.
      at new DOMException (node:internal/per_context/domexception:53:5)
      at MessagePort.<anonymous> (/Users/jwatkins/Development/Temporal/TypeScriptSDK/samples-typescript/hello-world/node_modules/@temporalio/worker/lib/workflow/workflow-worker-thread.js:92:20),
  workflowExists: true
}

3. Note that activation don't fail and no DataCloneError is thrown if the MyCustomError class is modified to extend Error (the workflow task still fails, which is expected), as follows:

class MyCustomError extends Error {
  public readonly x: any;

  constructor(public message: string) {
    super(message);
    this.x = () => 1;
  }
}

Details

There are few factors contributing to this issue:

• The Workflow Worker automatically logs exceptions that bubble up out of the workflow function;
• The V8 Serialization can't serialize some type of objects (including functions), and throws a DataCloneError on encountering a non-serializable object.
• As the list of all sink calls is transferred from the worker thread to the main thread as a single message, the failure to encode a single object causes the whole operation to fail.

[jan-stehlik]

hey @mjameswh thanks for writing this up, we are experiencing the same behaviour. Do you have any idea when this might be picked up / prioritised? Thanks

[mjameswh]

@jan-stehlik If your workflow is throwing exceptions that results in a DataCloneError, then the proper thing to do is to intercept those exceptions and throw something that can be safely serialized using the V8 serialization algorithm, such as an ApplicationFailure or a class that extends Error.

This ticket is not about resolving the fact that throwing non-serializable objects results in a DataCloneError, as this is inherent to the V8's serialization algorithm and we have pretty much no control on it, but rather to make it easier for users to understand these situations and figure out where the non-serializable object come from.