aws-vpc-flow-log-appender icon indicating copy to clipboard operation
aws-vpc-flow-log-appender copied to clipboard

Decorator Function Error

Open Treello opened this issue 7 years ago • 3 comments

When i am streaming from firehose i call the function it says error. this is what i see in the logs.

 23:04:08 START RequestId: 867b3d2a-12a4-11e8-bf8e-1928728ff215 Version: $LATEST START RequestId: 867b3d2a-12a4-11e8-bf8e-1928728ff215 Version: $LATEST  23:04:08 2018-02-15T23:04:08.716Z 867b3d2a-12a4-11e8-bf8e-1928728ff215 Received 11375 records for processing 2018-02-15T23:04:08.716Z 867b3d2a-12a4-11e8-bf8e-1928728ff215 Received 11375 records for processing  23:04:18 2018-02-15T23:04:18.857Z 867b3d2a-12a4-11e8-bf8e-1928728ff215 Finished building ENI to Security Group Mappig and Extracting Records 2018-02-15T23:04:18.857Z 867b3d2a-12a4-11e8-bf8e-1928728ff215 Finished building ENI to Security Group Mappig and Extracting Records  23:04:18 2018-02-15T23:04:18.857Z 867b3d2a-12a4-11e8-bf8e-1928728ff215 Decorating 11375 records 2018-02-15T23:04:18.857Z 867b3d2a-12a4-11e8-bf8e-1928728ff215 Decorating 11375 records  23:04:46 END RequestId: 867b3d2a-12a4-11e8-bf8e-1928728ff215 END RequestId: 867b3d2a-12a4-11e8-bf8e-1928728ff215  23:04:46 REPORT RequestId: 867b3d2a-12a4-11e8-bf8e-1928728ff215 Duration: 38059.39 ms Billed Duration: 38100 ms Memory Size: 128 MB Max Memory Used: 129 MB REPORT RequestId: 867b3d2a-12a4-11e8-bf8e-1928728ff215 Duration: 38059.39 ms Billed Duration: 38100 ms Memory Size: 128 MB Max Memory Used: 129 MB  23:04:46 RequestId: 867b3d2a-12a4-11e8-bf8e-1928728ff215 Process exited before completing request RequestId: 867b3d2a-12a4-11e8-bf8e-1928728ff215 Process exited before completing request

Treello avatar Feb 15 '18 23:02 Treello

I think my logs might be coming in to large?

Treello avatar Feb 15 '18 23:02 Treello

To overcome this issue, Change the output(Delivery) kinesis stream variables ES buffer size (MB)* to 1 MB

santhosh244 avatar Feb 17 '18 05:02 santhosh244

^ worked for me. 3 MB (the default) resulted in too much data being passed back from Lambda after being decorated. Change it to 1 MB.

adclark1 avatar Oct 30 '18 17:10 adclark1