WatchTree (zookeeper) doesn't send event when value of a existing child key changed

[YanXiaoping]

[root@localhost ~]# zookeepercli --servers=localhost:2181 -c ls /nsre/Units GISU-5

And I have a program watch on nsre/Units/.

1. add a new key/value pair zookeepercli --servers=localhost:2181 --debug=1 -c create /nsre/Units/GISU-6 test zookeepercli --servers=localhost:2181 -c ls /nsre/Units GISU-5 GISU-6 My program can receive event from watch channel as expected.
2. change value of a existing key zookeepercli --servers=localhost:2181 --debug=1 -c set /nsre/Units/GISU-6 changeit zookeepercli --servers=localhost:2181 --debug=1 -c get /nsre/Units/GISU-6 changeit My program can not receive any event! I think this is not expected behavior of WatchTree, right?

It works if etcd is used as store backend.

[YanXiaoping]

@abronan any comment to this?

[danielkrainas]

I dug into this and it looks to be on ZK's end, it doesn't send the event. I modified go-zookeeper to log the res.xid from ZK and then created a watcher with libkv, then modified the target key and no event was seen by libkv or go-zookeeper.

test log:

2016/10/21 21:51:18 Connected to 0.0.0.0:2181
2016/10/21 21:51:18 Authenticated: id=96802055589330982, timeout=10000
2016/10/21 21:51:18 Re-submitting `0` credentials after reconnect
2016/10/21 21:51:18 recv loop started
2016/10/21 21:51:18 res.xid 1
2016/10/21 21:51:18 res.xid 2
2016/10/21 21:51:18 res.xid 3
2016/10/21 21:51:18 res.xid 4
2016/10/21 21:51:18 res.xid 5
2016/10/21 21:51:18 res.xid 6
2016/10/21 21:51:18 res.xid 7
watch loop   <- libkv/zookeeper started its watcher loop
2016/10/21 21:51:21 res.xid -2
2016/10/21 21:51:24 res.xid -2
2016/10/21 21:51:28 res.xid -2   <- I modified the key and only heartbeats were sent by ZK
2016/10/21 21:51:31 res.xid -2
// -2 continues...

I was using the official zookeeper docker image with default settings. I'm also not a ZK expert so maybe I missed a configuration setting that would have altered my results.