etcd-mesos
etcd-mesos copied to clipboard
tasks not being cleaned up
I'm seeing some scenario's where the etcd-server tasks are not being cleaned up.
The initial setup was the following on DC/OS: 1 etcd task 3 etcd-server tasks
I cleaned the etcd records in zookeeper to be able to restart etcd.
Then I edited the app definition in DC/OS to change the MEM_LIMIT and CPU_LIMIT parameters and deployed the changes. As expected this deployed 3 new etcd-server tasks with the new memory and cpu limitations. However, it didn't stop the old etcd-server tasks after having deployed the 3 new ones. This brought me to the following situation: 1 etcd task 3 old etcd-server tasks still running, and "etcdctl member list" still showing them as the only members 3 new etcd-server tasks, with "http://etcd.marathon.mesos:5191/members" showing these new tasks as the only members
At this point the etcd records in zookeeper only show the new etcd-server task ips.
Then, in an idle hope to get all the tasks killed and start from a clean slate, I suspended the etcd service in DC/OS. Which resulted in the etcd task getting killed, but the etcd-server tasks kept running. I would've expected a suspend operation to kill all the etcd tasks.
So it seems like the etcd-server tasks are never cleaned up.
Using the 0.0.3 package of etcd-mesos in DC/OS We are running DC/OS on CoreOS on Amazon AWS