ol-infrastructure
ol-infrastructure copied to clipboard
mitodl
Infrastructure automation code for use by MIT Open Learning
### Expected Behavior If forum can't talk to its mongodb or opensearch backends, the app should crash / stop outright. Not enter a funky state where the ASG / LB...
``` [Unit] Description="Obtain an exclusive lock and run database migrations at startup." Requires=network-online.target After=consul-template.service [Service] Type=oneshot ExecStart=/usr/local/bin/consul lock -timeout=6h -verbose lms_migrations /edx/bin/edxapp-migrate-lms ExecStart=/usr/local/bin/consul lock -timeout=6h -verbose cms_migrations /edx/bin/edxapp-migrate-cms RemainAfterExit=True StandardOutput=journal...
### Description/Context The OCW team would like to keep the old OCW origin for an indeterminate amount of time. Currently it is an EC2 instance running Nginx and keeping all...
### Description/Context ``` 4:04 PM sar Yeah think as long as we don't have the grant definitions, then we should be good as those are the statements that are causing...
### Description/Context During the course of https://github.com/mitodl/ol-infrastructure/issues/2518 I got to thinking of how to best do these migrations / upgrade for consul clusters in a safe manner. It occurs to...
### Description/Context The OCW Plone service has been shut down. This is a placeholder to remind us to clean up the related AWS resources. ### Plan/Design - [ ] Terminate...
### Description/Context Doing the current add new keycloak user performance art for each engineer seems pointlessly grindy. There should be a way to have engineering users automatically added as a...
### Description/Context The idea is to alert when the queue is backed up. ### Plan/Design I don't honestly know what's involved here or whether or not celery-monitoring is a good...
### Description/Context In a recent MITx post mortem, we learned that we had 17K tasks running at once. This represents a critical condition we should be alerting on. meeting notes:...
### Description/Context Right now the management of secrets for devs is entirely ad-hoc. This lends itself to unfortunate incidents where monstrously huge .env files contain all the secrets and linger...