amazon-redshift-monitoring
amazon-redshift-monitoring copied to clipboard
Lambda function timing out while publishing cloudwatch metrics
I deployed CloudWatch using SAM and I can see the data being fetched from Redshift Cluster within 1 minute, however the Lambda function still doesn't complete after running for 5 minutes,.
Lambda logs report: Executing Redshift Diagnostic Query: WLMQuerySlotCountWarning Publishing 24 CloudWatch Metrics END RequestId: b054f912-b614-11e8-aa9e-d5eb851e7827 REPORT RequestId: b054f912-b614-11e8-aa9e-d5eb851e7827 Duration: 300005.52 ms Billed Duration: 300000 ms Memory Size: 192 MB Max Memory Used: 32 MB 2018-09-11T22:52:41.248Z b054f912-b614-11e8-aa9e-d5eb851e7827 Task timed out after 300.01 seconds
Cloudwatch Log report:
22:47:42 Executing Redshift Diagnostic Query: WLMQuerySlotCountWarning 22:47:42 Publishing 24 CloudWatch Metrics Publishing 24 CloudWatch Metrics 22:52:41 END RequestId: b054f912-b614-11e8-aa9e-d5eb851e7827 END RequestId: b054f912-b614-11e8-aa9e-d5eb851e7827 22:52:41 REPORT RequestId: b054f912-b614-11e8-aa9e-d5eb851e7827 Duration: 300005.52 ms Billed Duration: 300000 ms Memory Size: 192 MB Max Memory Used: 32 MB 22:52:41 2018-09-11T22:52:41.248Z b054f912-b614-11e8-aa9e-d5eb851e7827 Task timed out after 300.01 seconds 22:52:41 Pushing metrics to CloudWatch failed: exception ('Connection aborted.', error(1, 'Operation not permitted')) 22:52:41 /var/task/redshift_monitoring.py:249: SyntaxWarning: name 'debug' is assigned to before global declaration 22:52:41 global debug
The function hang at below statement:
try:
cw.put_metric_data(
Namespace='Redshift',
MetricData=put)
Verified the IAM Role has put permissions as suggested, also tried giving full access to the Role and it did not help. Please suggest.