hackforla
Epic (Devops): Deploy develop Branch to Staging Environment in AWS
Overview
Epic summary: As a development team, we want to deploy the develop branch to a staging environment in AWS to test the latest changes in an environment that closely mirrors production.
Acceptance Criteria:
- Ensure a fully automated deployment pipeline is configured for a staging environment in AWS.
- The pipeline should handle source control, continuous integration, environment provisioning, deployment, and monitoring.
- The pipeline is documented so that any new team members can easily read docs and look at a chart to understand the flow.
- A staging environment is deployed to AWS via Hack for LA's incubator account, following their required processes and procedures
Details
The develop branch will be used to test new code changes before merging to the main branch, which should deploy to a production environment.
For now, this epic will focus on the staging deployment only. The next step after this for devops is to deploy the main branch to production, then set it up so that whenever changes are made to the main branch, it is automatically deployed through terraform/github actions/AWS.
This epic is an overview of what generally needs to be done to achieve our goal. A devops engineer that works on this will need to break it down into more specific stories and action items, define their own tasks as well as track their own progress. It will require an ability to work independently and make sound decisons on behalf of the team.
Action Items
User Story 1: Create Staging Environment Infrastructure in AWS
- As a DevOps engineer, I want to define and provision a staging environment in AWS using infrastructure-as-code (IAC) so that deployments are consistent and replicable.
Acceptance Criteria:
- Use Terraform (and/or other IAC tools if necessary) to define staging infrastructure.
- Provision an isolated VPC, EC2 instances, or Lambda functions, depending on the application's architecture.
- Ensure environment mirrors the production environment as closely as possible.
Tasks
- [x] Write Terraform/CloudFormation templates to define AWS resources (e.g., VPC, ECS, RDS, S3, etc.).
- [x] Ensure security groups, IAM roles, and permissions are correctly set.
- [ ] Verify successful provisioning and connectivity between components.
User Story 2: Configure Deployment Pipeline to Staging
As a DevOps engineer, I want to automate the deployment process to AWS when the develop branch is merged into or updated, so that the application is consistently deployed to the staging environment.
Acceptance Criteria:
- Configure deployment triggers via GitHub Actions.
- Deploy the application to AWS ECS, EKS, or EC2 as per the infrastructure setup.
- Ensure rollback functionality is available in case of deployment failure.
Tasks:
- [x] Set up the CI/CD pipeline to deploy artifacts from the build phase to the AWS staging environment.
- [x] Define environment variables and secrets using AWS Systems Manager or Secrets Manager.
- [ ] Implement rollback strategy (e.g., using AWS CodeDeploy or Terraform).
User Story 3: Document devops Process
As a developer, I should be able to read documentation about the devops and deployments process so that I can have an understanding of how it works.
Acceptance Criteria:
- Our project's mkdocs should have a page about how AWS/incubator deployments are set up
- Mkdocs should have a page about our github actions related to the deployment pipelines
- There should be a diagram to easily visualize how the different AWS services are structured together
Tasks:
- [x] Write documentation about the AWS deployments
- [x] Write documentation about the github actions used for deployment
- [ ] Create an AWS service diagram
Resources/Instructions
- Resources - CivicTechJobs Docs
- https://hackforla.github.io/CivicTechJobs/developer/devops/
- https://github.com/hackforla/incubator
- https://developer.hashicorp.com/terraform
- https://docs.github.com/en/actions
We have this up: https://stage.civictechjobs.org/ But no one has any information about it
Update 10/07/2024:
- seems there is a CTJ stage deployment up on HfLA's incubator AWS account
- There are AWS files located in
./aws/in the codebase, and some github action workflows
Related issues that need to be processed:
@LoTerence Issue looks good, Moving it to prioritized backlog - until someone is ready to pick it up.
Hey @RSkuma - adding you here for visibility. Lets use this issue to keep track of updates regarding deployment to Incubator. Feel free to own this issue and remove/change anything in the description you feel is necessary. I wrote it using ChatGPT so its not 1-to-1 exactly what we need to do 😅 Thanks!!
Updates:
- Successfully deployed a staging environment to our personal AWS accounts
- AWS incubator deployment blocked by devops support - they are prioritizing VRMS because it's a live application. Should hopefully be free by mid-January.
- Ryan writing Terraform scripts in progress
I can verify the CTJ stage environment RDS database instance is connected and fully functional. You can test by going to:
- https://civictechjobs-stage.vrms.io/api/opportunities/
- https://civictechjobs-stage.vrms.io/admin/
The fact that these urls are accessible means:
- the stage database is working properly
- the stage db is connected with the stage django server
- stage server and stage db is successfully deployed
Hi @RSkuma could you please provide updates
- Progress
- Blockers
- Availability
- ETA