Ambry can't be put in a Zookeeper chroot

[siepkes]

Not really a big issue but I found out Zookeeper chroot's can't be used with Ambry. For example using the following connection string to chroot Ambry in Zookeeper in /ambry-misc:

zookeeper-1.foo.bar:2181/ambry-misc

will cause errors with Ambry storage nodes, the Helix bootstrap tool and probably a slew of other things.

As a workaround ACL's can be used instead.

[pnarayanan]

@siepkes thanks for bringing this to our attention. I assume you set this as part of clustermap.dcs.zk.connect.strings config? What errors did you see in the logs, could you paste them here?

[siepkes]

For example when using the HelixBootstrapUpgradeTool tool with the following arguments:

$ java -Dlog4j.configuration=file:/etc/ambry/log4j.properties \
    -cp /opt/ambry/ambry.jar \
    com.github.ambry.clustermap.HelixBootstrapUpgradeTool \
    --hardwareLayoutPath /etc/ambry/HardwareLayout.json \
    --partitionLayoutPath /etc/ambry/PartitionLayout.json \
    --clusterNamePrefix "" \
    --localDc dc-nl1 \
    --maxPartitionsInOneResource 3 \
    --zkLayoutPath /etc/ambry/ZookeeperLayout.json

With the following ZookeeperLayout.json which uses a Zookeeper chroot:

{
  "zkInfo": [
    {
      "datacenter": "dc-nl1",
      "zkConnectStr": "zookeeper-1.foo.nl:2181/ambry-misc,zookeeper-2.foo.nl:2181/ambry-misc,zookeeper-3.foo.nl:2181/ambry-misc"
    }
  ]
}

(The hardware and partition layout just contain a single partition and a single disk)

Results in the following error:

Associating static Ambry cluster "ambry-misc" with cluster"ambry-misc" in Helix
Exception in thread "main" org.I0Itec.zkclient.exception.ZkNoNodeException: org.apache.zookeeper.KeeperException$NoNodeException: KeeperErrorCode = NoNode for /
        at org.I0Itec.zkclient.exception.ZkException.create(ZkException.java:47)
        at org.I0Itec.zkclient.ZkClient.retryUntilConnected(ZkClient.java:893)
        at org.apache.helix.manager.zk.ZkClient.getChildren(ZkClient.java:209)
        at org.I0Itec.zkclient.ZkClient.getChildren(ZkClient.java:601)
        at org.apache.helix.manager.zk.ZKHelixAdmin.getClusters(ZKHelixAdmin.java:704)
        at com.github.ambry.clustermap.HelixBootstrapUpgradeUtil.initializeAdminsAndAddCluster(HelixBootstrapUpgradeUtil.java:207)
        at com.github.ambry.clustermap.HelixBootstrapUpgradeUtil.updateClusterMapInHelix(HelixBootstrapUpgradeUtil.java:174)
        at com.github.ambry.clustermap.HelixBootstrapUpgradeUtil.bootstrapOrUpgrade(HelixBootstrapUpgradeUtil.java:110)
        at com.github.ambry.clustermap.HelixBootstrapUpgradeTool.main(HelixBootstrapUpgradeTool.java:141)
Caused by: org.apache.zookeeper.KeeperException$NoNodeException: KeeperErrorCode = NoNode for /
        at org.apache.zookeeper.KeeperException.create(KeeperException.java:111)
        at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
        at org.apache.zookeeper.ZooKeeper.getChildren(ZooKeeper.java:1532)
        at org.apache.zookeeper.ZooKeeper.getChildren(ZooKeeper.java:1560)
        at org.I0Itec.zkclient.ZkConnection.getChildren(ZkConnection.java:112)
        at org.apache.helix.manager.zk.ZkClient$3.call(ZkClient.java:212)
        at org.apache.helix.manager.zk.ZkClient$3.call(ZkClient.java:209)
        at org.I0Itec.zkclient.ZkClient.retryUntilConnected(ZkClient.java:883)
        ... 7 more