Make Microservice creation dependent on the ApplicationContext

[charmpitz]

Feature Request

Is your feature request related to a problem? Please describe.

I believe that the way Nestjs is bootstraped could be improved by allowing the definition of the container before the definition of the application.

Take as an example a ConfigService that would read all the variables in .env in order to use them as options to define a new Nestjs Microservice while having the following code, how would you do it?

async function bootstrap() {
    const app = await NestFactory.createMicroservice(AppModule, {
        transport: Transport.RMQ,
        options: {
            urls: [
                'amqp://guest:guest@localhost:5672',
            ],
            queue: 'test',
            queueOptions: { durable: true },
        },
    });
    ...
}

At the moment only 2 solutions come in mind:

• define an ApplicationContext to be able to get the ConfigService from the container to use it for instantiating the microservice.
• define a service that should be instantiated manually before without DependencyInjection

Describe the solution you'd like

I believe that this could be done by instantiating the ApplicationContext that loads up the Container and then use the ApplicationContext as a parameter for the NestFactory.createMicroservice method.

Teachability, Documentation, Adoption, Migration Strategy

In order to support backwards compatibility you could even allow both the current way in case the paramenter a Module or the following one in case it is an instance of an ApplicationContext

async function bootstrap() {
    const app = await NestFactory.createApplicationContext(AppModule);

    const config = app.get(ConfigService);

    const microservice = await NestFactory.createMicroservice(app, {
        transport: Transport.RMQ,
        options: {
            urls: [
                config.amqpUrl,
            ],
            queue: config.queue,
            queueOptions: { durable: true },
        },
    });
    ...
}

What is the motivation / use case for changing the behavior?

• I believe this could improve the quality of the code.
• Could be useful in order to add a single instance of a LoggerService via DependecyInjection and also allow setting config options on it in order to call external services like Logstash etc.

[peteyycz]

@kamilmysliwiec is this a planned feature?

[scientiststwin]

I found this solution ->

const app = await NestFactory.create(AppModule, {
    logger: new Logger(),
})
const configService = app.get<ConfigService>(ConfigService)
app.connectMicroservice<MicroserviceOptions>({
    transport: Transport.RMQ,
    options: {
        urls: [amqp://${configService.get<string>('rabbitmq.host')}:${configService.get<number> 
         ('rabbitmq.port')}],
        queue: 'authentication',
        queueOptions: {
              durable: false,
          },
    },
})
app.startAllMicroservices(() => {
logger.log('Microservice is listening...')
})

this is work (usual MICROSERVICES) ! but this appear not work for websockets

[anatvas101]

I`m implementing Nest.js powered microservice and I need to do two things. Firstly, I need to get a configuration service in my main.ts file, and pass it as a constructor param to a custom transport strategy. Secondly, I need to do some bootstrapping actions in my service after microservice was bootstrapped. The issue I faced seems related to this issue: when I`m using app = await NestFactory.create(AppModule) and then app.connectMicroservice({strategy: myCustomStrategy}) - onApplicationBootstrap event has not fired. But when I`m using appContext = await NestFactory.createApplicationContext(AppModule) and then microservice = await NestFactory.createMicroservice(appContext, {strategy: myCustomStrategy}) - there is an error 'metatype is not a constructor' raised at NestFactoryStatic.createMicroservice. So for now I don`t understand, how I can pass correct environment config and catch the bootstrapping event simultaneously. Please, help me to figure out, if it can be done somehow.

[dlferro]

I found this solution ->

const app = await NestFactory.create(AppModule, {
    logger: new Logger(),
})
const configService = app.get<ConfigService>(ConfigService)
app.connectMicroservice<MicroserviceOptions>({
    transport: Transport.RMQ,
    options: {
        urls: [amqp://${configService.get<string>('rabbitmq.host')}:${configService.get<number> 
         ('rabbitmq.port')}],
        queue: 'authentication',
        queueOptions: {
              durable: false,
          },
    },
})
app.startAllMicroservices(() => {
logger.log('Microservice is listening...')
})

this is work (usual MICROSERVICES) ! but this appear not work for websockets

The one bit that becomes a sticking point with this example is that lifecycle methods do not work.

[artem-itdim]

I found this solution ->

const app = await NestFactory.create(AppModule, {
    logger: new Logger(),
})
const configService = app.get<ConfigService>(ConfigService)
app.connectMicroservice<MicroserviceOptions>({
    transport: Transport.RMQ,
    options: {
        urls: [amqp://${configService.get<string>('rabbitmq.host')}:${configService.get<number> 
         ('rabbitmq.port')}],
        queue: 'authentication',
        queueOptions: {
              durable: false,
          },
    },
})
app.startAllMicroservices(() => {
logger.log('Microservice is listening...')
})

this is work (usual MICROSERVICES) ! but this appear not work for websockets

The one bit that becomes a sticking point with this example is that lifecycle methods do not work.

Yes, but you can call the app.init method manually

  app.startAllMicroservices(async () => {
    // Calls the Nest lifecycle events.
    await app.init()
    logger.log('Microservice is listening...')
  })

[rluvaton]

Is this gonna be solved or stay this way with the current workarounds, in my opinion this is pretty trivial usage

[MickL]

@kamilmysliwiec Is this targeted for some version of Nest? What do you think of the two solutions/workarounds (#1 #2)? Do they have any downsides? E.g. #1 would create everything twice which uses unnecessary ressources and #2 would create a http app with Express unnecessarily running?

Personally I would prefer the following method as it does not create a Express server and all lifecycle methods work as expected:

async function bootstrap(): Promise<void> {
  // TODO: Remove when the following is fixed https://github.com/nestjs/nest/issues/2343
  const appContext    = await NestFactory.createApplicationContext(AppModule);
  const configService = appContext.get(ConfigService);
  // TODO End

  const app = await NestFactory.createMicroservice<MicroserviceOptions>(AppModule, {
    transport: Transport.NATS,
    options:   {
      url: configService.get<string>('nats.url'),
    },
  });

app.listen(() => {
    console.log('Microservice is listening');
  });

  // TODO: Remove when the following is fixed https://github.com/nestjs/nest/issues/2343
  appContext.close();
  // TODO End
}

[Matt007]

If your config parser only relies on process.env you could simply instantiate the config parser. This is what I've done:

server-config.ts

import { Transport, TcpClientOptions, RmqOptions } from '@nestjs/microservices';

export default () => ({
  Server: {
    transport: Transport[process.env['Server.transport']],
    tcp: {
      transport: Transport.TCP,
      options: {
        port: parseInt(process.env['Server.tcp.port']) || undefined,
        host: process.env['Server.tcp.host'] || undefined
      }
    } as TcpClientOptions,
    rmq: {
      transport: Transport.RMQ,
      options: {
        urls: typeof process.env['Server.rmq.urls'] != 'undefined' ? process.env['Server.rmq.urls'].split(',') : '',
        queue: process.env['Server.rmq.queue'],
        queueOptions: {
          durable: process.env['Server.rmq.queueOptions.durable']
        }
      } as RmqOptions
    }
  }
});

main.ts

import ServerConfig from './config/server-config.ts'
const serverConfig = ServerConfig().Server;

const app = await NestFactory.createMicroservice<MicroserviceOptions>(
  AppModule, {
    transport: Transport[String(serverConfig.transport).toUpperCase()], 
    options: serverConfig[String(serverConfig.transport).toLowerCase()]
  },
);

[Scrip7]

@kamilmysliwiec Is this targeted for some version of Nest? What do you think of the two solutions/workarounds (#1 #2)? Do they have any downsides? E.g. #1 would create everything twice which uses unnecessary ressources and #2 would create a http app with Express unnecessarily running?

Personally I would prefer the following method as it does not create a Express server and all lifecycle methods work as expected:

async function bootstrap(): Promise<void> {
  // TODO: Remove when the following is fixed https://github.com/nestjs/nest/issues/2343
  const appContext    = await NestFactory.createApplicationContext(AppModule);
  const configService = appContext.get(ConfigService);
  // TODO End

  const app = await NestFactory.createMicroservice<MicroserviceOptions>(AppModule, {
    transport: Transport.NATS,
    options:   {
      url: configService.get<string>('nats.url'),
    },
  });

app.listen(() => {
    console.log('Microservice is listening');
  });

  // TODO: Remove when the following is fixed https://github.com/nestjs/nest/issues/2343
  appContext.close();
  // TODO End
}

I use multiple TypeORM DB connections in my application. After trying this temporary fix, none of them could establish a connection to the database. It seems like it actually kills all the TypeORM DB connections because the temporary appContext is being closed! In order to fix that we have to provide keepConnectionAlive: true to TypeOrmModule options.

@Module({
  imports: [
    TypeOrmModule.forRoot({
      // ...
      keepConnectionAlive: true,
    }),
  ],
})
export class AppModule {}

So with that being said, If your application is heavily dependent on multiple database connections, I recommend manually taking process.env.YOUR_VAR with a proper typing approach instead of ConfigService.

[Scrip7]

This is how I added this feature to my application by using the env-var package. https://gist.github.com/Scrip7/63a159057e733284fdc963f3f2d2acdc

transport-config.ts:

import { NestMicroserviceOptions } from '@nestjs/common/interfaces/microservices/nest-microservice-options.interface';
import { MicroserviceOptions, RedisOptions, RmqOptions, TcpOptions, Transport } from '@nestjs/microservices';
import { get } from 'env-var';

export class TransportConfig {
  public static getOptions(): NestMicroserviceOptions & MicroserviceOptions {
    switch (get('TRANSPORT_LAYER').required().asString()) {
      case 'TCP':
        return TransportConfig.tcp();
      case 'REDIS':
        return TransportConfig.redis();
      case 'RMQ':
        return TransportConfig.rmq();

      default:
        throw new Error('Unsupported transport layer.');
    }
  }

  private static tcp(): TcpOptions {
    return {
      transport: Transport.TCP,
      options: {
        host: get('TCP_HOST').asString(),
        port: get('TCP_PORT').asPortNumber(),
      },
    };
  }

  private static redis(): RedisOptions {
    return {
      transport: Transport.REDIS,
      options: {
        url: get('REDIS_URL').required().asString(),
      },
    };
  }

  private static rmq(): RmqOptions {
    return {
      transport: Transport.RMQ,
      options: {
        urls: [
          {
            hostname: get('RMQ_HOST').required().asString(),
            port: get('RMQ_PORT').required().asPortNumber(),
            username: get('RMQ_USER').required().asString(),
            password: get('RMQ_PASS').required().asString(),
          },
        ],
        queue: get('RMQ_QUEUE').required().default('...').asString(),
        queueOptions: {
          durable: false,
        },
      },
    };
  }
}

main.ts

import { NestFactory } from '@nestjs/core';
import { MicroserviceOptions } from '@nestjs/microservices';
import { AppModule } from './app.module';
import { Logger } from '@nestjs/common';
import { TransportConfig } from './transport-config.ts'; // <-- Import it

async function bootstrap() {
  const logger = new Logger('Main');

  const app = await NestFactory.createMicroservice<MicroserviceOptions>(
    AppModule,
    TransportConfig.getOptions(), // <-- Use this
  );

  app.listen(() => logger.log('Microservice is listening'));
}
bootstrap();

.env

# TCP, REDIS, RMQ
TRANSPORT_LAYER=TCP

# TCP
TCP_HOST=localhost
TCP_PORT=3000

# Redis
REDIS_URL=redis://localhost:6379

# RabbitMQ
RMQ_HOST=localhost
RMQ_PORT=5672
RMQ_USER=guest
RMQ_PASS=guest
RMQ_QUEUE=your-lovely-queue

[SET001]

@kamilmysliwiec Is this targeted for some version of Nest? What do you think of the two solutions/workarounds (#1 #2)? Do they have any downsides? E.g. #1 would create everything twice which uses unnecessary ressources and #2 would create a http app with Express unnecessarily running?

Personally I would prefer the following method as it does not create a Express server and all lifecycle methods work as expected:

async function bootstrap(): Promise<void> {
  // TODO: Remove when the following is fixed https://github.com/nestjs/nest/issues/2343
  const appContext    = await NestFactory.createApplicationContext(AppModule);
  const configService = appContext.get(ConfigService);
  // TODO End

  const app = await NestFactory.createMicroservice<MicroserviceOptions>(AppModule, {
    transport: Transport.NATS,
    options:   {
      url: configService.get<string>('nats.url'),
    },
  });

app.listen(() => {
    console.log('Microservice is listening');
  });

  // TODO: Remove when the following is fixed https://github.com/nestjs/nest/issues/2343
  appContext.close();
  // TODO End
}

this seems work fine for me except that I need to close context, before creating a microservice:

  const appContext = await NestFactory.createApplicationContext(AppModule);
  const configService = appContext.get(ConfigService);
  const { host, tcpPort, name } = configService.get<ServiceServerConfig>('server')

  appContext.close();
  const app = await NestFactory.createMicroservice<MicroserviceOptions>(
    AppModule,
    {
      transport: Transport.TCP,
      options: {
        port: tcpPort,
        host
      }
    },
  );

  app.listen(() => console.log(`${name} microservice is listening`));

@Scrip7 this may be a solution for DB connection problem you mentioned

[mekwall]

Passing in the full AppModule to createApplicationContext makes it twice as slow to start and can be problematic when dealing with database connectivity. I tweaked @Scrip7's workaround by passing in ConfigModule directly. It's fast enough and doesn't introduce any race conditions.

import config from "./someConfigFile.ts";

async function bootstrap(): Promise<void> {
  // Custom logger
  const logger = createLogger("microservice");
  // TODO: Remove when the following is fixed https://github.com/nestjs/nest/issues/2343
  const appContext = await NestFactory.createApplicationContext(
    ConfigModule.forRoot({
      load: [config],
    }),
    {
      // Pass in your logger here or just set it to false if you want the
      // temporary application context to be silent
      logger
    }
  );
  const config = appContext.get(ConfigService);
  // TODO End

  const port = config.get(`services.thisService.port`);
  const app = await NestFactory.createMicroservice<MicroserviceOptions>(
    AppModule,
    {
      transport: Transport.TCP,
      options: { port },
    },
  );
  await app.listenAsync();
  // TODO: Remove when the following is fixed https://github.com/nestjs/nest/issues/2343
  // Close the temporary app context since we no longer need it
  appContext.close();
  // TODO End
  
  logger.info(`Microservice is listening on port ${port}`);
}

[flashpaper12]

Any updates on this issue? 👀

[dgradwell-ams]

I too, would like this feature.. and not because I need to load some config beforehand, but I use a lot of custom transport strategies that have injected dependencies. It would be nice not to have to create an express instance and connectMicroservice() or create two app contexts that load the entire dependency tree and establish connections, etc.

[vlad-rz]

It's been almost 3 years now. Can we work it out?