cloud-platform
cloud-platform copied to clipboard
Implement longer term storage for our logs
Background
We have 30 days of logs, and want 13 months. AWS meeting recommended implementing Warm and Cold storage for the ES cluster.
Approach
- instances we use are quite old, change to newer. Might be useful to upgrade the instance type, as storage not CPU is our main cost. (done)
- upgrade release to at least ES 7.9 (done)
- add the options of UltraWarm and Cold
Which part of the user docs does this impact
The process should be completely transparent for our users
Definition of done
- [ ] cluster upgraded
- [ ] newer instance types
- [ ] another team member has reviewed
- [ ] logs still come in
- [ ] curator job edited to cleanup after 1yr
- [ ] infra terraform code edited for the new config
Note: scope of this is not to fix the json parser, that is the topic of a separate story