Dead Letter for Synchronously Processed Commands/Events

[lifinsky]

Description

Currently, Ecotone provides a built-in Dead Letter mechanism that relies on DBAL for storing failed messages in the database. However, this mechanism is designed primarily for asynchronous message processing.

In synchronous execution flows — such as HTTP request handling — there is no automatic way to leverage the existing DLQ functionality when an exception occurs. This presents several issues:

• If a synchronously executed command or event fails (e.g., during an HTTP request), it is lost unless explicitly caught and handled.
• Developers are forced to implement custom exception handling logic to manually store failed messages, which leads to inconsistency with Ecotone’s built-in DLQ.
• Moving all such commands/events to an asynchronous channel just to use the DLQ is often impractical.

Use Case

• Webhook processing: Some webhooks should not be retried but must still be logged in the DLQ for later analysis.

• Business-critical HTTP requests: Commands triggered by HTTP should not be retried automatically but should still be stored in the DLQ for debugging.

• Exception handling for synchronous workflows: Ensuring that failed messages are safely persisted in the database instead of being lost.

[dgafka]

I understand the need for Commands where we trigger an action from HTTP Controller. I don't see the use case for synchronous Events however. As synchronous Events are result of synchronous Commands, so they are part of same transaction, therefore in my opinion they should fail together and be treated as one.

What I can propose is custom ErrorChannel on the level of Command Bus, which then can be used for different Commands as needed.

It would be defined as follows:

#[ErrorChannel("dbal_dead_letter")]
interface CommandBusWithDeadLetter extends CommandBus
{
}

What do you think?

[lifinsky]

I understand the need for Commands where we trigger an action from HTTP Controller. I don't see the use case for synchronous Events however. As synchronous Events are result of synchronous Commands, so they are part of same transaction, therefore in my opinion they should fail together and be treated as one.

What I can propose is custom ErrorChannel on the level of Command Bus, which then can be used for different Commands as needed.

It would be defined as follows:

#[ErrorChannel("dbal_dead_letter")] interface CommandBusWithDeadLetter extends CommandBus { } What do you think?

It's a good solution, but it would be even more flexible to define it for specific command handlers. And of course, it's important to have the ability to trigger the processing again via a console command — ideally allowing the handler to reprocess it asynchronously.

[dgafka]

It's doable for Message Handlers, however I don't see how this solution can be stable at that level. So suppose we put ErrorChannel flag on top of one of three synchronous Event Handlers (Transaction is wrapping all of them). Now Handler having Error Channel will fail with SQL exception, let's say Error Channel will handle the error, yet the next Event Handler try to trigger any other SQL within scope of same transaction, and as a result this will fail (because transaction is already marked for rollback).

This is exactly the same issue, which we discussed with nested retries. Therefore those operations are not atomic on that level, and can't be consider as such. That's why Error Channel is actually at the beginning of the flow, before transaction management.

Unless I am missing something, that has to be done on the level of Message Gateway (Command Bus) to be stable for all scenarios.

[lifinsky]

It's doable for Message Handlers, however I don't see how this solution can be stable at that level. So suppose we put ErrorChannel flag on top of one of three synchronous Event Handlers (Transaction is wrapping all of them). Now Handler having Error Channel will fail with SQL exception, let's say Error Channel will handle the error, yet the next Event Handler try to trigger any other SQL within scope of same transaction, and as a result this will fail (because transaction is already marked for rollback).

This is exactly the same issue, which we discussed with nested retries. Therefore those operations are not atomic on that level, and can't be consider as such. That's why Error Channel is actually at the beginning of the flow, before transaction management.

Unless I am missing something, that has to be done on the level of Message Gateway (Command Bus) to be stable for all scenarios.

On root CommandHandler as option

[dgafka]

It's doable for Message Handlers, however I don't see how this solution can be stable at that level. So suppose we put ErrorChannel flag on top of one of three synchronous Event Handlers (Transaction is wrapping all of them). Now Handler having Error Channel will fail with SQL exception, let's say Error Channel will handle the error, yet the next Event Handler try to trigger any other SQL within scope of same transaction, and as a result this will fail (because transaction is already marked for rollback). This is exactly the same issue, which we discussed with nested retries. Therefore those operations are not atomic on that level, and can't be consider as such. That's why Error Channel is actually at the beginning of the flow, before transaction management. Unless I am missing something, that has to be done on the level of Message Gateway (Command Bus) to be stable for all scenarios.

On root CommandHandler as option

That still does not solve the problem. As it will handle the exception on the Command Handler level, and then try to commit the transaction on Command Bus level (because error was handled) and that will fail.

[lifinsky]

@dgafka Can’t we roll back the current transaction and save to the dead letter table (error_messages) by placing an around interceptor for the ErrorChannel between the transactional interceptor and our command handler?

[dgafka]

@lifinsky transaction interceptors are around interceptors, therefore they work in this manner:

beginTransaction();
   $methodInvocation->proceed(); // Whatever is intercepted
commit();

And they are set up on Command Bus level, not on Command Handler level. Therefore it looks like this:

So it does not trigger on the Command Handler level, but just after the moment you trigger Command Bus.

Now if we would put ErrorChannel on Command Handler level (around interceptor for Command Handler), then it would happen down the stack, so it couldn't really happen before transaction interceptor:

---

So yea we can do custom error channel, however it has to be on CommandBus level then to work for all cases, and then using precedence we can adjust the positions against transaction interceptor:

[lifinsky]

I was guided by the idea of ​​not registering handlers for nested commands and supporting a common ecosystem of attributes for methods.

[dgafka]

This is now available as Enterprise feature starting from 1.258.1

Cheers :)