CloudWatch: S3 Requests metrics alerts

[sj-williams]

Background

We have a user requesting the ability to configure alerts based on S3 bucket access (GET/POST requests, success/fail/auth etc).

Out of the box, CloudWatch provides Storage metrics for object size and count metrics. Additional Request metrics can be enabled

https://github.com/ministryofjustice/cloud-platform-terraform-monitoring/blob/main/templates/cloudwatch-exporter.yaml#L60-L75

Actual metrics available: https://docs.aws.amazon.com/AmazonS3/latest/userguide/metrics-dimensions.html#s3-request-cloudwatch-metrics

Check viability of this request, approach would require similar method to that used by other team with api gateway. https://github.com/ministryofjustice/cloud-platform-environments/blob/32e3094ae2965abb69559c1ff5adf6b84c2904db/namespaces/live.cloud-platform.service.justice.gov.uk/hmpps-integration-api-prod/resources/api_gateway.tf#L225

Additionally, we don't currently have an s3 CW backed Grafana dashboard setup, could take a look at this as well.

Proposed user journey

Approach

Which part of the user docs does this impact

Communicate changes

• [ ] post for #cloud-platform-update
• [ ] Weeknotes item
• [ ] Show the Thing/P&A All Hands/User CoP
• [ ] Announcements channel

Questions / Assumptions

Definition of done

• [ ] readme has been updated
• [ ] user docs have been updated
• [ ] another team member has reviewed
• [ ] smoke tests are green
• [ ] prepare demo for the team

Reference

How to write good user stories

[sj-williams]

Discussed request and implementation approach with user. Confirmed that they can proceed with enabling custom metrics for S3 and applying required terraform resources to enable s3 request based alerts.