amazon-kinesis-agent icon indicating copy to clipboard operation
amazon-kinesis-agent copied to clipboard

Please add ability to specify filePattern that includes all subdirectories

Open et304383 opened this issue 7 years ago • 2 comments

I may be doing this wrong, but the following does not work:

{
  "cloudwatch.emitMetrics": false,
  "flows": [
    {
      "filePattern": "/var/log/myapp/**/*log",
      "deliveryStream": "myapp_logs"
    }
  ]
}

I want to see logs /var/log/myapp/*log and /var/log/myapp/subfolder/*log and so forth to be sent to Firehose. This doesn't appear to work though.

et304383 avatar Mar 17 '17 10:03 et304383

There's even a TODO in the code: https://github.com/awslabs/amazon-kinesis-agent/blob/51c60e07b7f2069a1424aa651fa09a2b46fdc721/src/com/amazon/kinesis/streaming/agent/tailing/SourceFile.java#L47

mvanbaak avatar Jul 12 '17 09:07 mvanbaak

Any update on this one? Also hit this case with some docker containers in /var/lib/docker/containers/<hash>/<hash>-json.log and I was wondering why this pattern didn't seem to work: /var/lib/docker/containers/*/*-json.log

prandelicious avatar Jan 25 '19 11:01 prandelicious