aws-vpc-flow-log-appender
aws-vpc-flow-log-appender copied to clipboard
Decorator Function Error
When i am streaming from firehose i call the function it says error. this is what i see in the logs.
23:04:08 START RequestId: 867b3d2a-12a4-11e8-bf8e-1928728ff215 Version: $LATEST START RequestId: 867b3d2a-12a4-11e8-bf8e-1928728ff215 Version: $LATEST 23:04:08 2018-02-15T23:04:08.716Z 867b3d2a-12a4-11e8-bf8e-1928728ff215 Received 11375 records for processing 2018-02-15T23:04:08.716Z 867b3d2a-12a4-11e8-bf8e-1928728ff215 Received 11375 records for processing 23:04:18 2018-02-15T23:04:18.857Z 867b3d2a-12a4-11e8-bf8e-1928728ff215 Finished building ENI to Security Group Mappig and Extracting Records 2018-02-15T23:04:18.857Z 867b3d2a-12a4-11e8-bf8e-1928728ff215 Finished building ENI to Security Group Mappig and Extracting Records 23:04:18 2018-02-15T23:04:18.857Z 867b3d2a-12a4-11e8-bf8e-1928728ff215 Decorating 11375 records 2018-02-15T23:04:18.857Z 867b3d2a-12a4-11e8-bf8e-1928728ff215 Decorating 11375 records 23:04:46 END RequestId: 867b3d2a-12a4-11e8-bf8e-1928728ff215 END RequestId: 867b3d2a-12a4-11e8-bf8e-1928728ff215 23:04:46 REPORT RequestId: 867b3d2a-12a4-11e8-bf8e-1928728ff215 Duration: 38059.39 ms Billed Duration: 38100 ms Memory Size: 128 MB Max Memory Used: 129 MB REPORT RequestId: 867b3d2a-12a4-11e8-bf8e-1928728ff215 Duration: 38059.39 ms Billed Duration: 38100 ms Memory Size: 128 MB Max Memory Used: 129 MB 23:04:46 RequestId: 867b3d2a-12a4-11e8-bf8e-1928728ff215 Process exited before completing request RequestId: 867b3d2a-12a4-11e8-bf8e-1928728ff215 Process exited before completing request
I think my logs might be coming in to large?
To overcome this issue, Change the output(Delivery) kinesis stream variables ES buffer size (MB)* to 1 MB
^ worked for me. 3 MB (the default) resulted in too much data being passed back from Lambda after being decorated. Change it to 1 MB.