Lifecycle hook in autoscaling group not recreated from codedeploy deployment group

[gunnarvelle]

When a codedeploy deployment group is created and it references an autoscaling group, a lifecycle hook is created in the autoscaling group. If this lifecycle hook accidentaly gets deleted, terraform does not recreate it because the deployment group is unchanged.

Terraform Version

11.2

Affected Resource(s)

• aws_autoscaling_group
• aws_codedeploy_deployment_group

Terraform Configuration Files

resource "aws_autoscaling_group" "instance" {
  name                      = "${var.env_name}-${var.component_name}"
  launch_configuration      = "${aws_launch_configuration.instance.name}"

  lifecycle {
    create_before_destroy = true
  }

  max_size                  = "${var.autoscaling_max_size}"
  min_size                  = "${var.autoscaling_min_size}"
  health_check_grace_period = "${var.autoscaling_grace_period}"
  health_check_type         = "${var.autoscaling_health_check_function}"
  vpc_zone_identifier       = [ "${split(",", var.private_subnets)}" ]

  tag {
    key                 = "Environment"
    value               = "${var.env_name}"
    propagate_at_launch = true
  }
}

resource "aws_autoscaling_attachment" "asg_public_elb" {
  autoscaling_group_name = "${aws_autoscaling_group.instance.id}"
  elb                    = "${aws_elb.instance.id}"
}

resource "aws_codedeploy_app" "instance" {
  name = "${var.env_name}.${var.component_name}"
}

resource "aws_codedeploy_deployment_group" "instance" {
  deployment_group_name  = "${var.env_name}.${var.component_name}"
  app_name               = "${aws_codedeploy_app.instance.name}"
  service_role_arn       = "${var.codedeploy_auth_role}"

  deployment_config_name = "CodeDeployDefault.OneAtATime"
  autoscaling_groups     = [ "${aws_autoscaling_group.instance.name}" ]

  trigger_configuration {
    trigger_events     = [ "DeploymentStart", "DeploymentSuccess", "DeploymentFailure", "DeploymentStop",
      "InstanceStart", "InstanceSuccess", "InstanceFailure" ]
    trigger_name       = "${var.component_name}-deployment"
    trigger_target_arn = "${var.deployment_notification_arn}"
  }
}

Expected Behavior

Terraform should have discovered the missing lifecycle hook, and recreated it.

Actual Behavior

Lifecycle hook still missing. I have to edit the deployment group using the admin console and save it with no changes. Then the lifecycle hook gets recreated by codedeploy.

Steps to Reproduce

1. terraform apply
2. Delete lifecycle hook from autoscaling group.
3. terraform apply

Important Factoids

None I am aware of.

[bflad]

Hi @gunnarvelle, in this case it sounds like Terraform is not managing a resource it does not know about. There are a few cases across AWS where API/UI calls will create other resources behind the scenes which then need to be separately managed in Terraform (Lambda functions automatically creating CloudWatch log groups as one example). In order to fully manage these, you must either create them initially with the other resources or import them after initial creation. Usually the API/UI will not complain if they already exist. See the documentation for aws_autoscaling_lifecycle_hooks if you want to manage these.

[gunnarvelle]

I can see it will be hard for terraform to manage something it does not know about, but in this case creating a lifecycle hook will not work. I managed to create such a hook using terraform, identical to the autogenerated save for the name which is on the form CodeDeploy-managed-automatic-launch-deployment-hook-application-some_id. Using my hook did not work, so it seems codedeploy need its own.

One thing which could solve this would be the ability to mark a resource with a flag telling terraform to save the resource, even if there are no changes. Such a flag on the codedeply_deployment_group should make aws restore the lifecycle hook without the need to include it in terraform.

[awartoft]

@gunnarvelle Did you ever solve this, I just ran into this problem and I have not found a decent way to solve it yet.

[gunnarvelle]

@macnibblet The only way to reinitialize the hook is to edit the application in codedeploy and save it without making changes. Then the lifecycle hook is refreshed or created if missing.

[JagoMar]

I assume that this issue doesn't have any other solution - I am experimenting this on my production environment.

[nicholaison]

Any solid solutions to this issue? I cannot restrict console access if this problem still exists :(

[hacking-robot]

Same issue here, after a blue/green rollback, the lifecycle is not recreated in the current ASG, I have to edit the app in codedeploy and hit save. That's crazy the numbers of issues there are in AWS with ASG and codedeploy.

[gfrankel97]

I also ran into this issue. Would love to see a resolution or (not manual) workaround. Is there an AWS CLI command that we can run to update the autoscaling group lifecycle hook? At least this way we can put it into a CI/CD system.

[nbari]

I am currently fixing this with something like

aws autoscaling describe-lifecycle-hooks --auto-scaling-group-name test_pipeline

From the deployment:

 aws deploy get-deployment-group --application-name test_pipeline --deployment-group-name test_pipeline | jq .[].autoScalingGroups

If they differ, then run update-deployment-group:

 aws deploy update-deployment-group --application-name test_pipeline --current-deployment-group-name test_pipeline

 aws deploy update-deployment-group --application-name test_pipeline --current-deployment-group-name test_pipeline --auto-scaling-groups test_pipeline

Wondering if there an easy way?

[atrakic]

I have experienced exactly the same but my aws_codedeploy_deployment_group resource was without trigger_configuration block.The solution was to open codedeployment group and press the "save changes" button - but without actually making any changes. After first deployment, asg lifecycle hook from codedeploy was generated back again.

[user9747]

Any updates on this? I can confirm that "saving without any changes" solution still works.

[rsaylor73]

This issue I have been having for the past 3 months and just found this solution. I have 4 autoscaling groups and confirm saving the code deploy application with no changes added the lifecycle hook back to my ASG. If I do a terraform plan terraform does not detect any change.