containers-roadmap icon indicating copy to clipboard operation
containers-roadmap copied to clipboard

ECS Service Discovery not respecting TTL when updating service

Open matthewduren opened this issue 5 years ago • 42 comments

Summary

When updating a service or otherwise scaling out ECS tasks for a service that uses Service Discovery, tasks are being stopped before reaching the TTL of the service discovery record(s).

Description

When updating a service or otherwise scaling out ECS tasks for a service that uses Service Discovery, tasks are being stopped before reaching the TTL of the service discovery record(s).

To reproduce - create an ECS service from a simple "hello world" type task definition that runs forever and does nothing. Set min healthy to 100, max to 200, count to 1. Setup service discovery and create a DNS record with a long TTL, say 300s. Update the service to use a new revision of the task definition (no changes to the task def needed), and note that the tasks are stopped before the TTL time is reached.

Expected Behavior

ECS Agent should remove the route53 record(s), and then wait to stop the tasks after the TTL duration has elapsed.

Observed Behavior

ECS Agent does not wait any additional time when stopping tasks for services that use service discovery.

Environment Details

Supporting Log Snippets

matthewduren avatar Oct 04 '18 18:10 matthewduren