copilot-cli
copilot-cli copied to clipboard
aws
Supporting Aurora Serverless v2
Since now Aurora Serverless v2 is generally available (https://aws.amazon.com/blogs/aws/amazon-aurora-serverless-v2-is-generally-available-instant-scaling-for-demanding-workloads/), are there any plans to support this improved version of Aurora Serverless as a built-in supported storage type?
Hi @tapicer. Thank you very much for bringing this up! Yes - we will be happy to support Aurora Serverless v2 as a built-in storage. Unfortunately for now the CFN support isn't available yet and we largely depend on that 🙏🏼 . Once we hear more from the Aurora team we'll be happy to put our update here!
+1 for this, ideally with a smooth upgrade path for those of us who are currently using v1 as a Copilot add-on
Apparently according to the public CloudFormation roadmap Serverless v2 is not coming soon (from https://github.com/aws/aws-cdk/issues/20197#issuecomment-1202459491) - and PostgreSQL 11.x feature set today is quite ancient (lots of current use SQL features don't work). This would seem a good reason to prioritize the regular RDS support requested in: https://github.com/aws/copilot-cli/issues/2984
Looks like this just dropped in CFN: https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-resource-rds-dbcluster.html#aws-resource-rds-dbcluster--examples--Creating_an_Amazon_Aurora_Serverless_v2_DB_cluster
The what's new: https://aws.amazon.com/about-aws/whats-new/2022/10/amazon-aurora-serverless-v2-supports-aws-cloudformation/ so we should be able to support this now! and everyone can also write their own addons/ until a new release of Copilot is out to leverage Aurora serverless v2 🎉
Looks like this just dropped in CFN: https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-resource-rds-dbcluster.html#aws-resource-rds-dbcluster--examples--Creating_an_Amazon_Aurora_Serverless_v2_DB_cluster
The what's new: https://aws.amazon.com/about-aws/whats-new/2022/10/amazon-aurora-serverless-v2-supports-aws-cloudformation/ so we should be able to support this now! and everyone can also write their own
addons/until a new release of Copilot is out to leverage Aurora serverless v2 🎉
That's great news! Do you plan to add some kind of command/script to allow the migration v1 to v2 copilot manifests? Or should we plan to do that manually?
Hi @tapicer, oh that's interesting, we hadn't thought of a command for migrating from v1 to v2. My understanding looking at the docs is that it can be done by restoring snapshots, but I haven't tested.
Our current plan was to change the copilot storage init command to generate from now on Serverless v2 templates rather than v1 to benefit from the elastic scaling of v2.
What's the baseline cost for v2? Have they actually solved the scale-to-zero problem, or does it still typically sit at 0.5-1 ACUs even with no queries and cost $40-$80/month "at rest"? That's a db.t3.medium or large at on-demand prices ...
Aurora Serverless v2 by default is now released in v1.23.0: https://github.com/aws/copilot-cli/releases/tag/v1.23.0! 🎉
Amazing news!
+1 for the request for an easy migration from v1. Currently my DB yml is at 'EngineVersion: '10.12'', EngineVersion: '10.12', Family: 'aurora-postgresql10', and no "ServerlessV2". Woudl LOVE to safetly migrate my current running DB to the latest configs.
@huanjani
Have they actually solved the scale-to-zero problem, or does it still typically sit at 0.5-1 ACUs even with no queries and cost $40-$80/month "at rest"?
How to lower the cost of running an Aurora v2 PostgreSQL cluster? I'm quickly realizing the $40-$80/mo. price to play without an option to lower that figure somehow. This is easy motivation to move away from copilot for anything but mega corp dollar projects.
Why must max capacity be 1? How about 0.25 or a more granular sliding scale?
morgan@mactop busmap (develop) $ copilot env deploy
Found only one environment, defaulting to: test
✔ Proposing infrastructure changes for the busmap-test environment.
- Creating the infrastructure for the busmap-test environment. [update rollback complete] [147.0s]
The following resource(s) failed to update: [AddonsStack].
- A CloudFormation nested stack for your additional AWS resources [update rollback complete] [131.2s]
The following resource(s) failed to update: [bmclusterDBCluster].
- The bmcluster Aurora Serverless v2 database cluster [update complete] [2.2s]
Resource handler returned message: "Serverless v2 maximum capacity 0.5
isn't valid. The maximum capacity must be at least 1.0. (Service: Rds
, Status Code: 400, Request ID: 0142ca18-a9b9-4886-b6d4-c80ba9daf1cc)"
(RequestToken: 92349a16-ae93-d500-73b6-4d9cb9a3f0e8, HandlerErrorCode
: InvalidRequest)
✘ deploy environment test: stack busmap-test did not complete successfully and exited with status UPDATE_ROLLBACK_COMPLETE
You don't need Aurora Serverless to run Copilot. I have a couple of Aurora Serverless v1 databases for dev apps that are much cheaper that I won't move to v2 at >2x the cost (no scale-blocking operations and v1 is working just fine). Most of my workloads aren't spiky, so we typically just use RDS as it's constant and cheaper.
Thanks for the reminder. Guess I should have passed the --serverless-version v1 option to copilot storage init. Still, I'm now thinking that is going to be more costly at rest than I would have thought and I'll be off to exploring other options until I find the sweet spot.