kyuubi icon indicating copy to clipboard operation
kyuubi copied to clipboard

[Bug] In the kerberos environment, the flink engine cannot be started, and the message "Delegation token is not supported" is displayed.

Open jiaoqingbo opened this issue 3 years ago • 2 comments

Code of Conduct

Search before asking

  • [X] I have searched in the issues and found no similar issues.

Describe the bug

In the kerberos environment, the flink engine cannot be started, and the message "Delegation token is not supported" is displayed.

Affects Version(s)

1.5.0

Kyuubi Server Log Output

10:11:48.766 WARN org.apache.kyuubi.client.KyuubiSyncThriftClient: TRenewDelegationTokenReq(sessionHandle:TSessionHandle(sessionId:THandleIdentifier(guid:47 F7 06 64 A5 BC 48 3D AA B0 8C 93 C0 91 3F 47, secret:E9 55 2B 1B B3 2E 44 99 BC 27 3C 69 11 D4 26 52)), delegationToken:SERUUwABETEwLjE5LjI5LjE3Mjo4MDIwPQAEb2NkcARvY2RwH3NwYXJrL29jZHAxNzIuYXNpYWlu^M
Zm8uY29tQG9jZHCKAX/Yd/tRigF//IR/UY4N2lEUJ5ccTLbM5cNPwuu6GFWKp5OZluQVSERGU19E^M
RUxFR0FUSU9OX1RPS0VOETEwLjE5LjI5LjE3Mjo4MDIwAA==) failed on engine side
org.apache.kyuubi.KyuubiSQLException: Delegation token is not supported
        at org.apache.kyuubi.KyuubiSQLException$.apply(KyuubiSQLException.scala:69) ~[kyuubi-common_2.12-1.5.0-incubating.jar:1.5.0-incubating]
        at org.apache.kyuubi.KyuubiSQLException$.apply(KyuubiSQLException.scala:81) ~[kyuubi-common_2.12-1.5.0-incubating.jar:1.5.0-incubating]
        at org.apache.kyuubi.client.KyuubiSyncThriftClient.sendCredentials(KyuubiSyncThriftClient.scala:244) ~[kyuubi-server_2.12-1.5.0-incubating.jar:1.5.0-incubating]

Kyuubi Engine Log Output

No response

Kyuubi Server Configurations

kyuubi-env.sh:
export JAVA_HOME=/usr/jdk64/jdk1.8.0_271
export HADOOP_CONF_DIR=/etc/hadoop/conf
export FLINK_HOME=/home/jiaoqingbo/flink-1.14.4
export HADOOP_CLASSPATH=`hadoop classpath`

kyuubi-defaults.conf
kyuubi.engine.type FLINK_SQL
kyuubi.authentication KERBEROS

Kyuubi Engine Configurations

No response

Additional context

The default values of kyuubi.credentials.hadoopfs.enabled and kyuubi.credentials.hive.enabled are both true,then LaunchEngine will call renewEngineCredentials() method。 However, FlinkTBinaryFrontendService does not implement the RenewDelegationToken method

Are you willing to submit PR?

  • [ ] Yes I am willing to submit a PR!

jiaoqingbo avatar Mar 30 '22 07:03 jiaoqingbo

GetDelegationToken、CancelDelegationToken、RenewDelegationToken Flink doesn't support to get the delegation token from the Yarn side now.

The FLIP-223 has explained this.

https://cwiki.apache.org/confluence/display/FLINK/FLIP-223%3A+Support+HiveServer2+Endpoint

a49a avatar Apr 28 '22 06:04 a49a

@deadwind4, could you please give the explanation of above reply? Why is the above bug related to get the delegation token from the Yarn side? You could refer to FLIP-211 for the delegation token in Flink.

KenjiFujima avatar May 28 '22 14:05 KenjiFujima