docker-cloudwatch icon indicating copy to clipboard operation
docker-cloudwatch copied to clipboard

Avoiding rate throttling

Open timkoopmans opened this issue 9 years ago • 2 comments

This seems like a good solution out of the box however might need to consider a way to batch apply events in the stream for anything with sub second logging interval (e.g. web server logs) as we'll inevitably hit rate limits on the API calls e.g.

image

timkoopmans avatar Apr 24 '15 02:04 timkoopmans

Just saw this comment :) https://github.com/nearform/cloudwatchlogs-stream/blob/master/index.js#L35

timkoopmans avatar Apr 24 '15 02:04 timkoopmans

Always hitting AWS SDK ThrottlingException. Added environment variable DEBUG="cloudwatchlogs" to check options. Batching with bulkIndex does not seem to solve the problem with a small number of log entries by any standards (docker-cloudwatch crashes almost immediately upon loading an nginx container and 2 dockerized nodeJS apps each displaying about 20 log entries over a loading time averaging 1 to 2 seconds).

image

jlchereau avatar Jun 08 '15 08:06 jlchereau