etcd-mesos icon indicating copy to clipboard operation
etcd-mesos copied to clipboard

tasks not being cleaned up

Open SlevinBE opened this issue 9 years ago • 0 comments

I'm seeing some scenario's where the etcd-server tasks are not being cleaned up.

The initial setup was the following on DC/OS: 1 etcd task 3 etcd-server tasks

I cleaned the etcd records in zookeeper to be able to restart etcd.

Then I edited the app definition in DC/OS to change the MEM_LIMIT and CPU_LIMIT parameters and deployed the changes. As expected this deployed 3 new etcd-server tasks with the new memory and cpu limitations. However, it didn't stop the old etcd-server tasks after having deployed the 3 new ones. This brought me to the following situation: 1 etcd task 3 old etcd-server tasks still running, and "etcdctl member list" still showing them as the only members 3 new etcd-server tasks, with "http://etcd.marathon.mesos:5191/members" showing these new tasks as the only members

At this point the etcd records in zookeeper only show the new etcd-server task ips.

Then, in an idle hope to get all the tasks killed and start from a clean slate, I suspended the etcd service in DC/OS. Which resulted in the etcd task getting killed, but the etcd-server tasks kept running. I would've expected a suspend operation to kill all the etcd tasks.

So it seems like the etcd-server tasks are never cleaned up.

Using the 0.0.3 package of etcd-mesos in DC/OS We are running DC/OS on CoreOS on Amazon AWS

SlevinBE avatar Oct 14 '16 09:10 SlevinBE