pino cloudwatch stop sending log when ulimit error

[samuraitruong]

Recently we have a bug in production that causes too many open file (ie ulimit). When this happens our application still works with some random error. however, the logs will stop sending to cloudwatch until we restart the server

Is there any config that lets pino keep re-try if there any network/tcp error.

[dbhowell]

Good question, and I'm not sure how it would resolve on a ulimit.

I assume the error log didn't make it to CloudWatch?

[samuraitruong]

@dbhowell

in general, when we have any error with tcp connection, then pino will stop working,

We are now fix the limit issue but when we run load testing and the application creating huge amount of logs then pino stop sending logs to cloudwatch even after we stop testing and the application back to normal traffic.

Is there anyway to configure re-try or keep alive incase of pino get an error?

[demian85]

I'm facing the same issue. I need to restart the server to resume cloudwatch logging.