docs.nestjs.com
docs.nestjs.com copied to clipboard
nestjs
Microservices interaction with external services
I'm submitting a...
[ ] Regression
[X ] Bug report
[ ] Feature request
[ ] Documentation issue or request
[ ] Support request => Please do not submit support request here, instead post your question on Stack Overflow.
Current behavior
When using nestjs as a NATS microservice, the nats payload (data), when using the @MessagePattern decorator, is always undefined.
@MessagePattern('sms.send')
test(data: any) {
console.log('function called OK');
console.log(data); // Allways undefined
}
In the provided example I can log "function called OK" but data is allways undefined.
Here is my main.ts
import { NestFactory } from '@nestjs/core';
import { Transport } from '@nestjs/microservices';
import { AppModule } from './app.module';
async function bootstrap() {
const app = await NestFactory.createMicroservice(AppModule, {
transport: Transport.NATS,
options: {
url: 'nats://nats:4222',
queue: 'sms',
},
});
app.listen(() => console.log('Microservice is listening'));
}
bootstrap();
I have tried both regular strings and json encoded objects as valid payloads.
Expected behavior
data should not be undefined
Minimal reproduction of the problem with instructions
What is the motivation / use case for changing the behavior?
Environment
Nest version: X.Y.Z
For Tooling issues:
- Node version: XX
- Platform:
Others:
Here you are, it is a verry basic example. https://github.com/purplefish32/nest-nats instructions are in the README file. By the way I used nest cli to generate the app.
The @EventPattern decorator, has exactly the same problem, I have updated my repo with an example.
A Ha!
I am on to something: if I change my simple nats emitter from
nats.publish('sms.send', JSON.stringify({ message: "JSON payload!" }));
to
nats.publish('sms.send', JSON.stringify({ data: "JSON payload!" }));
I recieve the proper paload.
This is both undocumented and IMHO not the correct way to handle a NATS payload. A NATS payload can be any type of string, not necesairily JSON, and the JSON can be any format.
My payloads are usualy formated like this :
{ "entity": "..." }
@kamilmysliwiec I would love to provide a pull request to fix this issue, but I am a Lerna noob and have no idea how to get the package.json of my test repo to use @nest/microservices from my fork instead of the one published on npm.
Is there a way to point directly to my github or do I have to use a private npm repo to publish my changes ?
Nest wraps every message exchanged by microservices with its own structure understandable by the framework (hence, data property is required). As far as I understand, you want to communicate with Nest microservice using an external service that is not built on top of Nest. We need to provide better docs for that, thanks for pointing this out.
I dont think it is a documentation issue, I think is the way that Nest microservices communicate via NATS is incorrect in that a NATS payload should be by default an arbitairy string not an opiniated JSON object.
IMHO, the nats-server should be initialised as a sting by default, or JSON via a new option
example:
const app = await NestFactory.createMicroservice(ApplicationModule, {
transport: Transport.NATS,
options: {
url: 'nats://localhost:4222',
json: true // New option
},
});
And in the case of a JSON paload : data used in @MessagePattern and @EventPattern should be the root object of the JSON payload, not the object.data
@MessagePattern('sms.send')
test(data: {} : null {
console.log(data)
}
I hope I am clear in my explanaition :)
Otherwise It means that every other non Nest NATS serivce must conform to the nestjs spec. which is a shame.
Nest needs to use particular JSON shape in order to stay cohesive between other transport layers. This JS object is serialized to string (stringified JSON) under the hood and sent through the network.
Otherwise It means that every other non Nest NATS serivce must conform to the nestjs spec. which is a shame.
Yes, this is exactly what it means and I don't think this is "a shame".
IMHO, the nats-server should be initialised as a sting by default, or JSON via a new option
This is something that we could potentially add in the future
@kamilmysliwiec
Are there any plans to allow custom shape? I'd like to migrate our services to NestJS, but we currently use this shape:
{
type, // Equivalent of NestJS "pattern"
appId, // Some data specific to our platform.
loginUserId, // Some data specific to our platform.
payload, // Equivalent of NestJS "data"
}
So far, this is really the only hurdle for our migration to NestJS. Awesome framework!
[EDIT]
Is there a way to add translation logic for messages? I tried using an Interceptor, but undefined is the only element in the context.args array. Looks like Nest is pulling out the data property before the Interceptor is called.
@goodoldneon what about passing your custom shape inside NestJS data? We need pattern to associate messages with handlers. You could then add interceptor that plucks data to avoid passing appId, type, and loginUserId to handler.
The problem is that our events are sent with the payload property, rather than data. Consequently, data is undefined in my MessageBrokerController.
I added an interceptor to rename the payload property to data, but context.args[0] is undefined. It seems as though there isn't a way to intercept the event before NestJS extracts data.
Eventually I want to migrate all of our services to NestJS (since NestJS is awesome ❤️), but in the meantime I need to add a layer that translates our existing event shape into the shape NestJS expects (pattern and data). I'm totally OK with NestJS being opinionated for event shape; I just need a way to make existing events adhere to NestJS's expectation.
@kamilmysliwiec any updates regarding laravel & nest integration using microservices ? :)
This article discusses the topic of serialization in depth. It focuses largely on NATS, but much of the discussion applies to all transporters: https://dev.to/nestjs/integrate-nestjs-with-external-services-using-microservice-transporters-part-1-p3
This article may also be of interest: https://dev.to/nestjs/part-1-introduction-and-setup-1a2l
Nest wraps every message exchanged by microservices with its own structure understandable by the framework (hence,
dataproperty is required). As far as I understand, you want to communicate with Nest microservice using an external service that is not built on top of Nest. We need to provide better docs for that, thanks for pointing this out. @kamilmysliwiec
Stumbled on the same issue where nest should act with external parties over a message queue and it took me quite a while to figure out how to use the @EventPattern() and @MessagePattern() decorators. +1 for adding a part in the documentation which goes over the expected data structure parsed by the mentioned decorators.
FYI i faced this issue, but fixed it by making my controller the event handler, instead of my service.
