Enable ZooKeeper client to establish connection in read-only mode

[massakam]

Motivation

If the system property readonlymode.enabled is set to true on a ZooKeeper server, read-only mode is enabled. Data can be read from the server in read-only mode even if that server is split from the quorum. https://zookeeper.apache.org/doc/current/zookeeperAdmin.html#Experimental+Options%2FFeatures

To connect to the server in read-only mode, the client must also allow read-only mode. The ZooKeeperClient class in the bookkeeper repository also has an option called allowReadOnlyMode. https://github.com/apache/bookkeeper/blob/15171e1904f7196d8e9f4116ab2aecdf582e0032/bookkeeper-server/src/main/java/org/apache/bookkeeper/zookeeper/ZooKeeperClient.java#L219-L222

However, even if this option is set to true, the connection to the server in read-only mode will actually fail. The cause is in the ZooKeeperWatcherBase class. When the ZooKeeperWatcherBase class receives the SyncConnected event, it releases clientConnectLatch and assumes that the connection is complete. https://github.com/apache/bookkeeper/blob/15171e1904f7196d8e9f4116ab2aecdf582e0032/bookkeeper-server/src/main/java/org/apache/bookkeeper/zookeeper/ZooKeeperWatcherBase.java#L128-L144

However, if the server is in read-only mode, it will receive ConnectedReadOnly instead of SyncConnected. This causes the connection to time out without being completed.

Changes

Modified the switch statement in the ZooKeeperWatcherBase class to release clientConnectLatch when ConnectedReadOnly is received if the allowReadOnlyMode option is true.

By the way, allowReadOnlyMode is never set to true in BookKeeper. So this change would be useless for BookKeeper. However, it is useful for Pulsar. Because Pulsar also uses ZooKeeperWatcherBase and needs to be able to connect to ZooKeeper in read-only mode. https://github.com/apache/pulsar/blob/cba1600d0f6a82f1ea194f3214a80f283fe8dc27/pulsar-metadata/src/main/java/org/apache/pulsar/metadata/impl/PulsarZooKeeperClient.java#L242-L244