[Bug] [Devops] When using Dinky dev version ,data sources that support HDFS to manage paimon
Open
dsanww
opened this issue 1 year ago
•
4 comments
Search before asking
[X] I had searched in the issues and found no similar issues.
What happened
When using Dinky dev version(commit:114cc18179bb857c93c34324b32438f97ef8af1c) ,data sources that support HDFS to manage paimon
An error was reported while submitting the paimon task:
Caused by: java.lang.RuntimeException: java.lang.NullPointerException
at org.apache.paimon.utils.HadoopUtils.readHadoopXml(HadoopUtils.java:220) ~[paimon-flink-1.18-1.0-20241009.jar:1.0-SNAPSHOT]
at org.apache.paimon.utils.HadoopUtils.addHadoopConfIfFound(HadoopUtils.java:175) ~[paimon-flink-1.18-1.0-20241009.jar:1.0-SNAPSHOT]
at org.apache.paimon.utils.HadoopUtils.getHadoopConfiguration(HadoopUtils.java:113) ~[paimon-flink-1.18-1.0-20241009.jar:1.0-SNAPSHOT]
at org.apache.paimon.catalog.CatalogContext.(CatalogContext.java:53) ~[paimon-flink-1.18-1.0-20241009.jar:1.0-SNAPSHOT]
at org.apache.paimon.catalog.CatalogContext.create(CatalogContext.java:73) ~[paimon-flink-1.18-1.0-20241009.jar:1.0-SNAPSHOT]
at org.apache.paimon.flink.FlinkCatalogFactory.createPaimonCatalog(FlinkCatalogFactory.java:81) ~[paimon-flink-1.18-1.0-20241009.jar:1.0-SNAPSHOT]
at org.apache.paimon.flink.action.ActionBase.initPaimonCatalog(ActionBase.java:78) ~[paimon-flink-1.18-1.0-20241009.jar:1.0-SNAPSHOT]
at org.apache.paimon.flink.action.ActionBase.(ActionBase.java:65) ~[paimon-flink-1.18-1.0-20241009.jar:1.0-SNAPSHOT]
at org.apache.paimon.flink.action.cdc.SynchronizationActionBase.(SynchronizationActionBase.java:77) ~[paimon-flink-1.18-1.0-20241009.jar:1.0-SNAPSHOT]
at org.apache.paimon.flink.action.cdc.SyncDatabaseActionBase.(SyncDatabaseActionBase.java:68) ~[paimon-flink-1.18-1.0-20241009.jar:1.0-SNAPSHOT]
at org.apache.paimon.flink.action.cdc.mysql.MySqlSyncDatabaseAction.(MySqlSyncDatabaseAction.java:109) ~[paimon-flink-1.18-1.0-20241009.jar:1.0-SNAPSHOT]
at org.apache.paimon.flink.action.cdc.mysql.MySqlSyncDatabaseActionFactory.createAction(MySqlSyncDatabaseActionFactory.java:52) ~[paimon-flink-1.18-1.0-20241009.jar:1.0-SNAPSHOT]
at org.apache.paimon.flink.action.cdc.mysql.MySqlSyncDatabaseActionFactory.createAction(MySqlSyncDatabaseActionFactory.java:31) ~[paimon-flink-1.18-1.0-20241009.jar:1.0-SNAPSHOT]
at org.apache.paimon.flink.action.cdc.SynchronizationActionFactoryBase.create(SynchronizationActionFactoryBase.java:45) ~[paimon-flink-1.18-1.0-20241009.jar:1.0-SNAPSHOT]
at org.apache.paimon.flink.action.cdc.SyncDatabaseActionFactoryBase.create(SyncDatabaseActionFactoryBase.java:47) ~[paimon-flink-1.18-1.0-20241009.jar:1.0-SNAPSHOT]
at org.apache.paimon.flink.action.ActionFactory.createAction(ActionFactory.java:87) ~[paimon-flink-1.18-1.0-20241009.jar:1.0-SNAPSHOT]
at org.apache.paimon.flink.action.FlinkActions.main(FlinkActions.java:43) ~[paimon-flink-1.18-1.0-20241009.jar:1.0-SNAPSHOT]
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[?:1.8.0_252]
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[?:1.8.0_252]
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:1.8.0_252]
at java.lang.reflect.Method.invoke(Method.java:498) ~[?:1.8.0_252]
at org.apache.flink.client.program.PackagedProgram.callMainMethod(PackagedProgram.java:355) ~[flink-dist-1.18.0.jar:1.18.0]
at org.apache.flink.client.program.PackagedProgram.invokeInteractiveModeForExecution(PackagedProgram.java:222) ~[flink-dist-1.18.0.jar:1.18.0]
at org.apache.flink.client.program.PackagedProgramUtils.getPipelineFromProgram(PackagedProgramUtils.java:145) ~[dinky-app-1.18-1.2.0-SNAPSHOT-jar-with-dependencies.jar:1.18.0]
at org.dinky.trans.dml.ExecuteJarOperation.getStreamGraph(ExecuteJarOperation.java:118) ~[dinky-app-1.18-1.2.0-SNAPSHOT-jar-with-dependencies.jar:?]
at org.dinky.trans.dml.ExecuteJarOperation.getStreamGraph(ExecuteJarOperation.java:84) ~[dinky-app-1.18-1.2.0-SNAPSHOT-jar-with-dependencies.jar:?]
at org.dinky.app.flinksql.Submitter.executeJarJob(Submitter.java:263) ~[dinky-app-1.18-1.2.0-SNAPSHOT-jar-with-dependencies.jar:?]
at org.dinky.app.flinksql.Submitter.submit(Submitter.java:142) ~[dinky-app-1.18-1.2.0-SNAPSHOT-jar-with-dependencies.jar:?]
at org.dinky.app.MainApp.main(MainApp.java:53) ~[dinky-app-1.18-1.2.0-SNAPSHOT-jar-with-dependencies.jar:?]
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[?:1.8.0_252]
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[?:1.8.0_252]
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:1.8.0_252]
at java.lang.reflect.Method.invoke(Method.java:498) ~[?:1.8.0_252]
at org.apache.flink.client.program.PackagedProgram.callMainMethod(PackagedProgram.java:355) ~[flink-dist-1.18.0.jar:1.18.0]
What you expected to happen
no error
How to reproduce
1、Set dinky data source paimon to HDFS and specify the hdfs configuration path
2、run paimon with flink application job on dinky
Hi @, Because this issue was marked as requiring detailed reproduction steps, no response was received for 10 days. The issue is now closed. If you have any questions, please comment and reply.
Hi @, 由于该 issue 被标记为需要提供详细复现步骤,却 10 天未收到回应。现关闭 issue,若有任何问题,可评论回复。