asgard icon indicating copy to clipboard operation
asgard copied to clipboard

rolling push to asg in vpc with target:elb fails

Open tjbaker opened this issue 10 years ago • 2 comments

Another issue, somewhat similar to bug #486

Using commit 5623fad as standalone jar or war on tomcat.

Rolling AMI push to an VPC ASG with immutable_metadata "target=elb" on the ASG associated subnets fails. If you remove the elb target from the subnets and let it assume target=ec2 the push works correctly.

Exception in thread "Task:Pushing ami-f3f2ff9a with package <<redacted>> into group <<redacted>> for app STG" com.amazonaws.AmazonServiceException: 
Status Code: 400, AWS Service: AmazonAutoScaling, 
AWS Request ID: b834fbba-a3cd-11e3-ac4b-0d61efd5ca20, 
AWS Error Code: ValidationError, 
AWS Error Message: The availability zones of the specified subnets and the AutoScalingGroup do not match
    at com.amazonaws.http.AmazonHttpClient.handleErrorResponse(AmazonHttpClient.java:767)
    at com.amazonaws.http.AmazonHttpClient.executeHelper(AmazonHttpClient.java:414)
    at com.amazonaws.http.AmazonHttpClient.execute(AmazonHttpClient.java:228)
    at com.amazonaws.services.autoscaling.AmazonAutoScalingClient.invoke(AmazonAutoScalingClient.java:2166)
    at com.amazonaws.services.autoscaling.AmazonAutoScalingClient.updateAutoScalingGroup(AmazonAutoScalingClient.java:1458)
    at com.netflix.asgard.AwsAutoScalingService$_updateAutoScalingGroup_closure49.doCall(AwsAutoScalingService.groovy:878)
    at com.netflix.asgard.TaskService.doWork(TaskService.groovy:85)
    at com.netflix.asgard.TaskService.runTask(TaskService.groovy:71)
    at com.netflix.asgard.AwsAutoScalingService.updateAutoScalingGroup(AwsAutoScalingService.groovy:880)
    at com.netflix.asgard.push.RollingPushOperation.prepareGroup(RollingPushOperation.groovy:135)
    at com.netflix.asgard.push.RollingPushOperation$_start_closure1.doCall(RollingPushOperation.groovy:85)
    at com.netflix.asgard.TaskService.doWork(TaskService.groovy:85)
    at com.netflix.asgard.TaskService$_startTask_closure1.doCall(TaskService.groovy:59)

tjbaker avatar Mar 04 '14 18:03 tjbaker

updating to see requests across the wire...

index 6795c7e..b76a81b 100644
--- a/grails-app/conf/Config.groovy
+++ b/grails-app/conf/Config.groovy
@@ -62,7 +62,7 @@ log4j = {
     warn 'org.codehaus.groovy.grails'

     // Set this to debug to watch the XML communications to and from Amazon
-    error 'org.apache.http.wire'
+    debug 'org.apache.http.wire'

Data sent to update ASG when target=EC2

DefaultCooldown=300
&HealthCheckGracePeriod=300
&AutoScalingGroupName=STG-Workers
&HealthCheckType=EC2
&Action=UpdateAutoScalingGroup
&DesiredCapacity=2
&AvailabilityZones.member.1=us-east-1a
&MaxSize=2
&VPCZoneIdentifier=<<redacted>>
&Version=2011-01-01
&TerminationPolicies.member.1=Default
&MinSize=1
&LaunchConfigurationName=STG-Workers-20140305152041

data sent to update the ASG when target=ELB

DefaultCooldown=300
&HealthCheckGracePeriod=300
&AutoScalingGroupName=STG-Web-ASG
&HealthCheckType=ELB
&Action=UpdateAutoScalingGroup
&DesiredCapacity=2
&AvailabilityZones.member.1=us-east-1a
&MaxSize=2
&Version=2011-01-01
&TerminationPolicies.member.1=Default
&MinSize=0
&LaunchConfigurationName=STG-Web-ASG-20140305145924

VPCZoneIdentifier is missing when target=ELB.

http://docs.aws.amazon.com/cli/latest/reference/autoscaling/update-auto-scaling-group.html

tjbaker avatar Mar 05 '14 20:03 tjbaker

When using the AWS UI console, attempting to update the ASG without specifying any subnets shows the warning "Auto Scaling groups created in a VPC must remain in a VPC" and you are not able to save. This scenario seems analogous to the asgard wire request above where VPCZoneIdentifier is not sent.

screen shot 2014-03-10 at 9 31 12 am

tjbaker avatar Mar 10 '14 13:03 tjbaker