Critical: chalice delete is deleting the entire custom domain name in API Gateway

[rafagan]

I've just run chalice delete to undeploy an api, and after that, chalice deleted my custom domain name from API Gateway entirely with all the mappings. It should delete only one mapping and not all of then. I've just lose the entire production stage mappings.

The bug happens when you have api_gateway_custom_domain configured for a domain in many other projects, running chalice delete for one of them will make you lose the custom domain name entirely.

That's a critical bug.

[rafagan]

Is chalice still maintained?

[jamesls]

I see what's happening here. Chalice is assuming it will always own the domain name, and that it's tied to a single rest api. So it will always create the domain name if necessary:

https://github.com/aws/chalice/blob/d6a9d8dab3d110ca0f8d63c5ba9bd95eb160e063/chalice/deploy/planner.py#L318-L325

and subsequently on deletion, it will delete the domain name if necessary:

https://github.com/aws/chalice/blob/d6a9d8dab3d110ca0f8d63c5ba9bd95eb160e063/chalice/deploy/sweeper.py#L184-L187

My initial thought was that the ideal behavior is if Chalice actually creates the domain name object as part of the initial deploy, it should be responsible for subsequently deleting it. However, if the domain name already exists (in which case Chalice doesn't need to create it as part of its initial deploy), then it should leave the domain name alone and not delete it.

However, I don't think this is the right solution for a couple reasons:

• This logic is not possible to implement in CFN/terraform so there will be inconsistencies in the backends.
• This logic is hard to track. Imagine you have 3 apps with the same custom domain but different base path mappings. You have to remember which of the three apps actually created the domain object and remember not to delete that app otherwise the entire domain name will be deleted.

Another option would be to never delete the domain name, but then we're leaking resources. Presumably at some point Chalice will have created the domain name for you in one of your apps, and we shouldn't leak resources we create.

I think the only reasonable option is have an explicit configuration option that instructs Chalice whether or not it should handle the creation/deletion of the domain. The idea is you'd set this option in only one of your Chalice apps, and disable this option in your other apps. That way it makes it clear by looking at your config file if deleting the app will also delete the domain name.

Does that seem reasonable?

[rafagan]

I think the best solution is:

Create flow:

1. Check if custom domain name exists, if not, create it
2. Create the mapping path

Delete flow:

1. Delete the mapping path
2. If there are another mapping paths, keep custom domain name alive
3. Else, delete it

[rafagan]

Any updates?

[knelk]

@rafagan @jamesls Did you find a solution to restore your other stages, after this accidental deletion happened?

[ddtraveller]

I had an issue with a chalice site where a new route wasn't routing. Eventually I added the code for the new route to an old route and it worked. I even deleted a route and recreated it and it worked but I couldn't add this new path, no matter the code.

In looking at API gateway, I noticed there were 3 apis with the same name for my project. In looking at the lambda, there were 3 triggers to api gateway when there should only have been one.

I ended up deleting the lambda, the apig and the custom domain api mappings. I had to remove the api gateway apis from within the api. At the main console, it just said "Too many requests".

I have 2 other chalice projects so I'm glad I didn't run chalice delete.