[Question] Organising email sending service

[kunal-mandalia]

Great work with this project, I'm new to DDD but your work (this repo and your website) has helped me get started. 👍

I've a couple of questions about how to organise services such as an email sending service. Suppose the use case is: when someone replies to my post I should receive an email.

1. Would the email service be defined in the infra layer and it'd subscribe to events e.g. postCreated?
2. Suppose we wanted to make sure that when creating a post both the post and the email must be sent or the request to create the post fails - how is that handled? I see some comments to Unit Of Work but not sure how that works in concrete terms.

[stemmlerjs]

Hey @kunal-mandalia, thanks for checking out the repo + site!

Great questions! I'll do my best to answer them both.

...when someone replies to my post I should receive an email. Would the email service be defined in the infra layer and it'd subscribe to events e.g. postCreated?

Yeah, you've got it! The most correct way I can answer this is that:

• We probably want a new subdomain in order to separate these concerns. I was thinking about doing this, but maybe you could take a stab at it if you feel like it. You know how we have modules/users and modules/forum in this project? I think sending emails, notifications, etc is neither the concern of the users nor the forum subdomain. So it would make sense to have a notifications subdomain. Potential use cases could be sendEmail, sendSlackNotification, sendPushNotification, etc (though we can focus on strictly email for now).
• In the adapter layer within the notifications subdomain, we can define an abstraction for an email service, perhaps an IEmailService. This is what we refer to from our application layer Use cases; just the abstraction, not the actual concrete class.
• In the infrastructure layer within the notifications subdomain, you can use any email provider you want (sendgrid, mailgun, etc) as long as the implementation implements the IEmailService adapter.
• Now, we can create some subscribers from within notifications, like AfterUserCreated subscriber in order to subscribe to domain events from other subdomains.

Suppose we wanted to make sure that when creating a post both the post and the email must be sent or the request to create the post fails - how is that handled? I see some comments to Unit Of Work but not sure how that works in concrete terms.

Value Objects can be used to ensure that we have a valid domain object in order to continue. Check out any of the Value Object classes (like UserEmail). Notice that we use Guard clauses to make sure that we can create a valid UserEmail to continue?

Here's an example from CommentText:

public static create (props: CommentTextProps): Result<CommentText> {
    // Null check
    const nullGuardResult = Guard.againstNullOrUndefined(props.value, 'commentText');
    
    // if it was null-y, we will return a failed "Result"
    // check out this article on why it's better to use a failed Result rather than to THROW errors.
    /// https://khalilstemmler.com/articles/enterprise-typescript-nodejs/handling-errors-result-class/
    if (!nullGuardResult.succeeded) {
      return Result.fail<CommentText>(nullGuardResult.message);
    }

    const minGuardResult = Guard.againstAtLeast(this.minLength, props.value);
    const maxGuardResult = Guard.againstAtMost(this.maxLength, props.value);

    if (!minGuardResult.succeeded) {
      return Result.fail<CommentText>(minGuardResult.message);
    }

    if (!maxGuardResult.succeeded) {
      return Result.fail<CommentText>(maxGuardResult.message);
    }

    return Result.ok<CommentText>(new CommentText(props));
  }

Suppose we wanted to make sure that when creating a post both the post and the email must be sent or the request to create the post fails.

In CreatePost.ts we actually do something like this already (though not with email)

If title: string and one of either text: string or link: string aren't present and valid in the CreatePostDTO, we return with a failed Result<T> if we can't create value objects (PostTitle, PostText and or PostLink) from them.

[kunal-mandalia]

@stemmlerjs thanks for taking the time to answer my questions. The notifications subdomain makes a lot of sense. I'm still not 💯 on how to transactionally perform actions across entities / services but I'll take a closer look at create post's (e.g. the execute handler) and value object guard clause in enforcing validation.

[stemmlerjs]

@kunal-mandalia Have you read this article yet? Let me know if that helps!

[StevePavlin]

For anyone else interested in solving this problem with this style of architecture. The way to transactionally ensure an email gets sent after a business transaction is the outbox pattern.

Jimmy Bogard goes into depth with it in that article. With a similar architecture to this one, I implemented a UnitOfWork class in shared/infra/database. The application use case layer begins a unit of work, makes modifications to domain entities, saves them to the repository, then adds rows to the outbox table based on the pending domain events on the AggregateRoot all within the same ACID SQL transaction.

I then use the sequelize afterCommit hook to immediately push the outboxed domain event into a message queue, which is consumed by subscribers (such as a notification subdomain), then deleted from the outbox table. This ensures at-least once delivery since the system may crash before removing the message from the outbox table, causing duplicate messages on the queue.

If the push onto the MQ fails, you can use a backoff pattern to try it again, or perhaps a backend worker queue running on an interval can ensure the message is eventually pushed. With this in mind, you should attempt to make your subscribers idempotent, since duplicate messages may invariant violations (tough to do with emails since most email service providers do not provide idempotent APIs).

@stemmlerjs does mention the outbox pattern adds a lot of complexity in his SOLID book, as you can see!

[ajoshi31]

@stemmlerjs I was also going through your repo and came across nearly same question where to keep the services like sending sms, emails, handling errors (I may use winston or some other library), so abstracting the usage away from subdomains in our application.

Just my thought instead of creating each subdomain for notification, error handling, monitoring etc, can't we use the shared section which is used across subdomains or say layers in each subdomain.

Any third party service we use for notification, error handling, monitoring can be kept at infra layer and can put port (interface) somewhere in shared section which can be used by any subdomain?

├── index.spec.ts
├── index.ts
├── module
│   ├── invoice
│   ├── search
│   └── user
│       ├── application
│       │   ├── mapper
│       │   ├── ports
│       │   │   ├── persistence
│       │   │   │   └── interface.txt
│       │   │   └── usecase
│       │   │       └── interface.txt
│       │   └── usecases-implementation
│       │       └── index.txt
│       ├── domain
│       │   └── index.txt // other domain events, vo, entities and domain services
│       ├── dtos
│       │   └── index.txt
│       └── infra
│           ├── http
│           │   └── user.controller.ts
│           └── repository
│               ├── inmem
│               └── post.model.ts
└── shared
    ├── core
    │   ├── error-handler
    │   │   └── error.ts
    │   ├── exceptions
    │   │   └── HttpException.ts
    │   └── logger
    │       └── logger.ts
    ├── infra
    │   ├── database
    │   │   ├── mongo-db
    │   │   │   ├── database.spec.ts
    │   │   │   └── database.ts
    │   │   └── mysql
    │   │       └── database.ts
    │   ├── http
    │   │   ├── express.ts
    │   │   ├── middleware
    │   │   │   ├── error.middleware.ts
    │   │   │   ├── index.ts
    │   │   │   └── requestLogger.middleware.ts
    │   │   └── router.ts
    │   ├── logger
    │   │   └── winston.ts
    │   ├── mailer
    │   │   └── sendgrid.ts
    │   └── messaging
    │       └── twilio.ts
    ├── port
    │   ├── logger.interface.ts
    │   ├── mailing.interface.ts
    │   └── messaging.interafce.ts
    └── utils
        ├── exitHandler.ts
        └── validateEnv.ts

[ydennisy]

@kunal-mandalia @stemmlerjs I think this would be a great addition of an example to the code base which integrates some 3p service!

[pbell23]

For anyone else interested in solving this problem with this style of architecture. The way to transactionally ensure an email gets sent after a business transaction is the outbox pattern.

Jimmy Bogard goes into depth with it in that article. With a similar architecture to this one, I implemented a UnitOfWork class in shared/infra/database. The application use case layer begins a unit of work, makes modifications to domain entities, saves them to the repository, then adds rows to the outbox table based on the pending domain events on the AggregateRoot all within the same ACID SQL transaction.

I then use the sequelize afterCommit hook to immediately push the outboxed domain event into a message queue, which is consumed by subscribers (such as a notification subdomain), then deleted from the outbox table. This ensures at-least once delivery since the system may crash before removing the message from the outbox table, causing duplicate messages on the queue.

If the push onto the MQ fails, you can use a backoff pattern to try it again, or perhaps a backend worker queue running on an interval can ensure the message is eventually pushed. With this in mind, you should attempt to make your subscribers idempotent, since duplicate messages may invariant violations (tough to do with emails since most email service providers do not provide idempotent APIs).

@stemmlerjs does mention the outbox pattern adds a lot of complexity in his SOLID book, as you can see!

Here is an other article which explains how the outbox pattern should be implemented : https://microservices.io/patterns/data/transactional-outbox.html .

I also looked through the code and this pattern seems essential to me for using domain events. Take this example, ddd-forum/src/modules/users/useCases/createUser/CreateUserUseCase.ts :

const userOrError: Result<User> = User.create({
        email, password, username,
      });

      if (userOrError.isFailure) {
        return left(
          Result.fail<User>(userOrError.getErrorValue().toString())
        ) as Response;
      }

      const user: User = userOrError.getValue();

      await this.userRepo.save(user);

      return right(Result.ok<void>())

    } catch (err) {
      return left(new AppError.UnexpectedError(err)) as Response;
    }

In this use case, when creating a new user, User.create static method will add a new domain event UserCreated. This event is listened by the forum module which will create a new member. It means that if

await this.userRepo.save(user);

fails, then you have a new member created without its associated user.

EDIT : I created a new issue specifically for that so that the title matches better the problem : #125