kyuubi icon indicating copy to clipboard operation
kyuubi copied to clipboard

Bump Zookeeper 3.6.3

Open pan3793 opened this issue 2 years ago • 3 comments

Why are the changes needed?

#2040 adds the test to verify the compatibility for Zookeeper Server 3.4, I will rebase this PR after #2040 get merged.

Close #1941

How was this patch tested?

  • [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible

  • [ ] Add screenshots for manual tests if appropriate

  • [ ] Run test locally before make a pull request

pan3793 avatar Mar 04 '22 17:03 pan3793

Codecov Report

Merging #2041 (eb7101a) into master (6a90b3e) will decrease coverage by 0.06%. The diff coverage is n/a.

@@             Coverage Diff              @@
##             master    #2041      +/-   ##
============================================
- Coverage     51.51%   51.44%   -0.07%     
  Complexity       13       13              
============================================
  Files           481      481              
  Lines         26761    26734      -27     
  Branches       3753     3750       -3     
============================================
- Hits          13785    13753      -32     
- Misses        11631    11632       +1     
- Partials       1345     1349       +4     
Impacted Files Coverage Δ
...rg/apache/kyuubi/events/EventHandlerRegister.scala 52.38% <0.00%> (-4.15%) :arrow_down:
...uthentication/LdapAuthenticationProviderImpl.scala 92.00% <0.00%> (-3.24%) :arrow_down:
...rg/apache/kyuubi/ctl/cmd/log/LogBatchCommand.scala 78.00% <0.00%> (-2.00%) :arrow_down:
...mon/src/main/scala/org/apache/kyuubi/Logging.scala 52.63% <0.00%> (-1.32%) :arrow_down:
.../engine/spark/session/SparkSQLSessionManager.scala 78.48% <0.00%> (-1.27%) :arrow_down:
...yuubi/server/metadata/jdbc/JDBCMetadataStore.scala 89.27% <0.00%> (-0.70%) :arrow_down:
...a/org/apache/kyuubi/service/TFrontendService.scala 91.17% <0.00%> (-0.30%) :arrow_down:
...in/scala/org/apache/kyuubi/config/KyuubiConf.scala 97.44% <0.00%> (-0.06%) :arrow_down:
...org/apache/kyuubi/operation/ExecuteStatement.scala 80.51% <0.00%> (ø)
...che/kyuubi/events/ServerEventHandlerRegister.scala 100.00% <0.00%> (ø)
... and 2 more

:mega: We’re building smart automated test selection to slash your CI/CD build times. Learn more

codecov-commenter avatar Mar 05 '22 10:03 codecov-commenter

thanks @SteNicholas, this pr seems not ready for review it's blocked by https://github.com/apache/incubator-kyuubi/pull/2040

yaooqinn avatar Mar 06 '22 15:03 yaooqinn

thanks @SteNicholas, this pr seems not ready for review it's blocked by #2040

Yea, marked this one as draft for now, I will address #2040 ASAP.

pan3793 avatar Mar 06 '22 15:03 pan3793

I verified this patch is ok in our environment.

Environment:

  • Kyuubi zookeeper client version: 3.6.3
  • Zookeeper server version: 3.4.13

Check items:

  • Connect normally using Zookeeper Service Discovery mode.
  • Kyuubi server and Spark engine start normally without any error log.
  • show databases is OK.
  • select statement is OK.

wForget avatar Sep 09 '22 02:09 wForget

run bin/kyuubi-zk-cli error:

Exception in thread "main" java.lang.NoClassDefFoundError: org/apache/commons/cli/ParseException
	at org.apache.zookeeper.ZooKeeperMain.<clinit>(ZooKeeperMain.java:108)
Caused by: java.lang.ClassNotFoundException: org.apache.commons.cli.ParseException
	at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
	at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
	at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:331)
	at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
	... 1 more

wForget avatar Sep 09 '22 02:09 wForget

ClassNotFoundException: org.apache.commons.cli.ParseException has been fixed by adding common-cli deps.

pan3793 avatar Sep 09 '22 04:09 pan3793

Thanks, merging to master

pan3793 avatar Sep 09 '22 04:09 pan3793

Reverted because breaks compatible of Zookeeper 3.4

Exception in thread "Thread-6" org.apache.kyuubi.KyuubiSQLException: Lock failed on path [/lock-test]
	at org.apache.kyuubi.KyuubiSQLException$.apply(KyuubiSQLException.scala:69)
	at org.apache.kyuubi.ha.client.zookeeper.ZookeeperDiscoveryClient.tryWithLock(ZookeeperDiscoveryClient.scala:171)
	at org.apache.kyuubi.ha.client.DiscoveryClientTests.$anonfun$$init$$13(DiscoveryClientTests.scala:139)
	at org.apache.kyuubi.ha.client.DiscoveryClientTests.$anonfun$$init$$13$adapted(DiscoveryClientTests.scala:137)
	at org.apache.kyuubi.ha.client.DiscoveryClientProvider$.withDiscoveryClient(DiscoveryClientProvider.scala:36)
	at org.apache.kyuubi.ha.client.DiscoveryClientTests.$anonfun$$init$$12(DiscoveryClientTests.scala:137)
	at java.lang.Thread.run(Thread.java:750)
Caused by: org.apache.zookeeper.KeeperException$UnimplementedException: KeeperErrorCode = Unimplemented for /lock-test/locks
	at org.apache.zookeeper.KeeperException.create(KeeperException.java:106)
	at org.apache.zookeeper.KeeperException.create(KeeperException.java:54)
	at org.apache.zookeeper.ZooKeeper.create(ZooKeeper.java:1734)
	at org.apache.curator.utils.ZKPaths.mkdirs(ZKPaths.java:291)
	at org.apache.curator.framework.imps.CreateBuilderImpl$11.call(CreateBuilderImpl.java:746)
	at org.apache.curator.framework.imps.CreateBuilderImpl$11.call(CreateBuilderImpl.java:723)
	at org.apache.curator.RetryLoop.callWithRetry(RetryLoop.java:109)
	at org.apache.curator.framework.imps.CreateBuilderImpl.pathInForeground(CreateBuilderImpl.java:720)
	at org.apache.curator.framework.imps.CreateBuilderImpl.protectedPathInForeground(CreateBuilderImpl.java:484)
	at org.apache.curator.framework.imps.CreateBuilderImpl.forPath(CreateBuilderImpl.java:474)
	at org.apache.curator.framework.imps.CreateBuilderImpl.forPath(CreateBuilderImpl.java:454)
	at org.apache.curator.framework.imps.CreateBuilderImpl.forPath(CreateBuilderImpl.java:44)
	at org.apache.curator.framework.recipes.locks.StandardLockInternalsDriver.createsTheLock(StandardLockInternalsDriver.java:54)
	at org.apache.curator.framework.recipes.locks.LockInternals.attemptLock(LockInternals.java:217)
	at org.apache.curator.framework.recipes.locks.InterProcessMutex.internalLock(InterProcessMutex.java:232)
	at org.apache.curator.framework.recipes.locks.InterProcessMutex.acquire(InterProcessMutex.java:108)
	at org.apache.curator.framework.recipes.locks.InterProcessSemaphoreV2.internalAcquire1Lease(InterProcessSemaphoreV2.java:344)
	at org.apache.curator.framework.recipes.locks.InterProcessSemaphoreV2.acquire(InterProcessSemaphoreV2.java:284)
	at org.apache.curator.framework.recipes.locks.InterProcessSemaphoreV2.acquire(InterProcessSemaphoreV2.java:246)
	at org.apache.curator.framework.recipes.locks.InterProcessSemaphoreMutex.acquire(InterProcessSemaphoreMutex.java:52)
	at org.apache.kyuubi.ha.client.zookeeper.ZookeeperDiscoveryClient.tryWithLock(ZookeeperDiscoveryClient.scala:163)
	... 5 more

pan3793 avatar Sep 09 '22 09:09 pan3793