MULTI TRANSPORT PROBLEM IN AWS LAMBDA

[Diego3310]

Hi, I am facing an issue using sync: true in my Pino multi transport configuration within an AWS Lambda function.

According to the Pino documentation, setting sync: true should make the transport operate synchronously. However, this does not seem to work as expected in my setup. Additionally, logs are not appearing correctly in CloudWatch or SQS, depending on the transport used.

Here’s the configuration I’m using for my transports:

import pino, { DestinationStream, TransportTargetOptions } from 'pino';

import { ILoggerTransportStrategy } from '../interfaces/transport-strategy.interface';
import config from '../../../../../../config/config';
import { TransportOptions } from '../../options/options';
import { GeneralTransportOptionsType, TransportOptionsType } from '../../../../../common';

/**
 * Clase que se encarga de configurar los transportes de logs
 */
export class MultiTransportStrategy implements ILoggerTransportStrategy {
  private targets: TransportTargetOptions[];

  constructor() {
    this.targets = [];
  }

  /**
   * Configura los transportes de logs
   * @param options - Opciones de configuración
   * @returns - El destino de los logs
   */
  configure(options: GeneralTransportOptionsType): DestinationStream {
    const targets: TransportOptionsType[] = config.get('log.targets') as TransportOptionsType[];

    if (targets.length === 0) {
      return pino.transport({
        targets: [TransportOptions.select('cloud', options)],
      });
    }

    for (const target of targets) {
      this.addTransport(TransportOptions.select(target, options));
    }

    const transports = pino.transport({
      targets: this.targets,
      sync: true,
    });

    this.targets = [];

    return transports;
  }

  /**
   * Añade un transporte a la lista de transportes
   * @param transport - Transporte a añadir
   */
  private addTransport(transport: TransportTargetOptions): void {
    if (transport === null || transport === undefined) {
      return;
    }

    this.targets.push(transport);
  }
}

/**
 * -  Opciones de transporte para la consola estándar.
 * @returns TransportSingleOptions
 */
export const cloudTargetOptions = (): TransportTargetOptions => {
  return {
    level: config.get('log.level'),
    target: 'pino/file',
    options: {
      destination: 1,
    },
  };
};

import build, { OnUnknown } from 'pino-abstract-transport';
import { SQSClient, SQSClientConfig, SendMessageCommand, SendMessageRequest } from '@aws-sdk/client-sqs';

/**
 * Opciones de configuración para el transporte SQS.
 */
export type SQSTransportOptionsType = {
  region?: string;
  queueUrl: string;
  messageGroupId?: string;
  sqsOptions?: SQSClientConfig;
};

/**
 * - Transporte personalizado para enviar logs a una cola de SQS
 * @param opts - Opciones de configuración
 *  @returns - Función que recibe un objeto y lo envía a la cola de SQS
 */
export default async function (opts: SQSTransportOptionsType): Promise<OnUnknown> {
  const { region = 'us-east-1', queueUrl, messageGroupId, sqsOptions } = opts;

  const sqsClient = new SQSClient({ region, ...sqsOptions });

  return build(async function (source) {
    for await (const obj of source) {
      const logMessage = JSON.stringify({ message: JSON.stringify(obj) });

      const params: SendMessageRequest = {
        QueueUrl: queueUrl,
        MessageBody: logMessage,
        MessageGroupId: messageGroupId,
      };

      try {
        const command = new SendMessageCommand(params);

        await sqsClient.send(command);
      } catch (error) {
        console.error('Error al enviar el mensaje a SQS:', error);
      }
    }
  });
}

Expected Behavior

The sync: true option should ensure synchronous operation for the transports in AWS Lambda. Logs should appear in the destinations (e.g., SQS or CloudWatch) during the Lambda invocation.

--

Actual Behavior

The sync: true option does not make the transports synchronous. Logs are not visible in SQS or CloudWatch.

[DominicRoyStang]

I'm experiencing the same issue. If this is intended behavior, I think docs should be clearer about the following points:

• Is it possible to have multiple transports, and have some/all of them be synchronous?
• Can any transport be made to run synchronously, or is there something that the transport creator needs to do to enable this functionality?

Other possibly related issue in pino-pretty: https://github.com/pinojs/pino-pretty/issues/504

[mcollina]

The core of this problem is that the execution model of lambda does not play well with multithreading. The solution to your issue is to:

const transport = pino.transport(...)

//right before returning the control to AWS lambda
transport.flushSync()

Hopefully this will get everything through.

Having said that, I would recommend against using asynchronous transports in lambda due to those issues.