cloudformation-coverage-roadmap icon indicating copy to clipboard operation
cloudformation-coverage-roadmap copied to clipboard

AWS::EC2::TransitGatewayRouteTable Tags changes should not require replacement

Open oliott opened this issue 3 years ago • 7 comments

Name of the resource

AWS::EC2::TransitGatewayRouteTable

Resource name

No response

Description

Adding or changing tags on a AWS::EC2::TransitGatewayRouteTable, causes replacement. Using the console and cli calls works fine. However cloudformation does not seem to handle it. This means that changing a tag in a Cloudformation template or CDK code could potential break entire networks or halt updates.

Expected behavior: Altering Tags definitions for a AWS::EC2::TransitGatewayRouteTable resource should trigger an in-place change, not replacement of the RouteTable.

Other Details

AWS::EC2::TransitGatewayRouteTable and AWS::EC2::TransitGatewayAttachment have the same issue as reported in: https://github.com/aws-cloudformation/cloudformation-coverage-roadmap/issues/531

oliott avatar May 06 '22 08:05 oliott

They must be working on this because now, tags update on a AWS::EC2::TransitGatewayRouteTable basicaly result in "Internal Failure".

bplessis-swi avatar Feb 08 '23 21:02 bplessis-swi

Oh no it's weirder, "cloudformation deploy" trigger an "Internal Failure", "cloudformation update-stack" trigger a resource replacement ...

bplessis-swi avatar Feb 09 '23 08:02 bplessis-swi

We just had an outage because of this unexpected behavior. Yes the docs clearly display on the cloudformation resource page that an update to tags would cause a replacement, but who would realistically look at every cloudformation resource doc when you are just trying to tag your resources for cost tracking purposes? Tagging should absolutely not be causing replacement of resources. Shame on AWS for not even responding/assigning this request. I'll probably open another just to see if we can get traction.

reaperharvest avatar Dec 21 '23 19:12 reaperharvest

i also accidentally trigger this error. doing a cost tagging update via CDK. and it caused update fail due to transit gateway tag update require replacement. oddly the cfn return a message that don't really mean anything. i run a cdk diff and review it to see only tag update.

Resource handler returned message: "The request must contain one or more of AddSubnetIds, RemoveSubnetIds, DnsSupport, Ipv6Support, ApplianceModeSupport (Service: Ec2, Status Code: 400, Request ID: 854c24a7-eaec-44b4-b97e-efeb18dfcb3b)" (RequestToken: 3a15b7b5-52e8-834b-cb52-43a8adc9932f, HandlerErrorCode: InvalidRequest)

jk2l avatar Mar 15 '24 02:03 jk2l

@prerna-p I disagree that this is an enhancement and not a bug. At least according to the documentation, the underlying TGW Route Table resource can be re-tagged without disruption. More importantly, many AWS customers will be associating things with the Route Table outside of CloudFormation (due to complications with how CloudFormation works and just because that's the nature of network configuration) which means that modifying tags here is a guarantee that things will break (resulting in a serious outage) for all of those customers. So yes it's a change in behavior, but only because the current behavior guarantees an outage.

(in the meantime, it would have been better if this resource simply rejected tag updates so it could fail-fast)

abatkin avatar Mar 15 '24 12:03 abatkin

@prerna-p I disagree that this is an enhancement and not a bug.

Didn't you mis-read the label change, they removed the enhancement part and indeed flagged this as a bug

bplessis-swi avatar Mar 15 '24 13:03 bplessis-swi

@prerna-p I disagree that this is an enhancement and not a bug.

Didn't you mis-read the label change, they removed the enhancement part and indeed flagged this as a bug

Sorry, you are correct, it is marked as a bug now, my apologies.

abatkin avatar Mar 15 '24 14:03 abatkin