seatunnel icon indicating copy to clipboard operation
seatunnel copied to clipboard

[Bug] [Hive Source] Invalid method name: 'get_table_req'

Open matianhe3 opened this issue 6 months ago • 15 comments

Search before asking

  • [X] I had searched in the issues and found no similar issues.

What happened

update to 2.3.6 , hive source error. 2.3.5 ok.

SeaTunnel Version

2.3.6

SeaTunnel Config

source {
  Hive {
    table_name = ""
    metastore_uri = ""
    hdfs_site_path = "/opt/apache-seatunnel/config/hdfs-site.xml"
    hive_site_path = "/opt/apache-seatunnel/config/hive-site.xml"
    result_table_name = source
    read_partitions = ["dt="${dt}]
    delimiter = ","
  }
}

Running Command

seatunnel.sh -c test.conf -i dt=2024-08-06

Error Exception

2024-08-07 11:10:59,057 ERROR [o.a.s.c.s.SeaTunnel           ] [main] - Exception StackTrace:org.apache.seatunnel.core.starter.exception.CommandExecuteException: SeaTunnel job executed failed
	at org.apache.seatunnel.core.starter.seatunnel.command.ClientExecuteCommand.execute(ClientExecuteCommand.java:211)
	at org.apache.seatunnel.core.starter.SeaTunnel.run(SeaTunnel.java:40)
	at org.apache.seatunnel.core.starter.seatunnel.SeaTunnelClient.main(SeaTunnelClient.java:34)
Caused by: org.apache.seatunnel.api.table.factory.FactoryException: ErrorCode:[API-06], ErrorDescription:[Factory initialize failed] - Unable to create a source for identifier 'Hive'.
	at org.apache.seatunnel.api.table.factory.FactoryUtil.createAndPrepareSource(FactoryUtil.java:101)
	at org.apache.seatunnel.engine.core.parse.MultipleTableJobConfigParser.parseSource(MultipleTableJobConfigParser.java:361)
	at org.apache.seatunnel.engine.core.parse.MultipleTableJobConfigParser.parse(MultipleTableJobConfigParser.java:209)
	at org.apache.seatunnel.engine.client.job.ClientJobExecutionEnvironment.getLogicalDag(ClientJobExecutionEnvironment.java:114)
	at org.apache.seatunnel.engine.client.job.ClientJobExecutionEnvironment.execute(ClientJobExecutionEnvironment.java:182)
	at org.apache.seatunnel.core.starter.seatunnel.command.ClientExecuteCommand.execute(ClientExecuteCommand.java:158)
	... 2 more
Caused by: java.lang.NoSuchMethodError: 'void org.apache.hadoop.hive.metastore.HiveMetaStoreClient.<init>(org.apache.hadoop.conf.Configuration)'
	at org.apache.seatunnel.connectors.seatunnel.hive.utils.HiveMetaStoreProxy.<init>(HiveMetaStoreProxy.java:110)
	at org.apache.seatunnel.connectors.seatunnel.hive.utils.HiveMetaStoreProxy.getInstance(HiveMetaStoreProxy.java:139)
	at org.apache.seatunnel.connectors.seatunnel.hive.utils.HiveTableUtils.getTableInfo(HiveTableUtils.java:41)
	at org.apache.seatunnel.connectors.seatunnel.hive.source.config.HiveSourceConfig.<init>(HiveSourceConfig.java:84)
	at org.apache.seatunnel.connectors.seatunnel.hive.source.config.MultipleTableHiveSourceConfig.parseFromLocalFileSourceConfig(MultipleTableHiveSourceConfig.java:52)
	at org.apache.seatunnel.connectors.seatunnel.hive.source.config.MultipleTableHiveSourceConfig.<init>(MultipleTableHiveSourceConfig.java:39)
	at org.apache.seatunnel.connectors.seatunnel.hive.source.HiveSource.<init>(HiveSource.java:43)
	at org.apache.seatunnel.connectors.seatunnel.hive.source.HiveSourceFactory.lambda$createSource$0(HiveSourceFactory.java:46)
	at org.apache.seatunnel.api.table.factory.FactoryUtil.createAndPrepareSource(FactoryUtil.java:113)
	at org.apache.seatunnel.api.table.factory.FactoryUtil.createAndPrepareSource(FactoryUtil.java:74)
	... 7 more

2024-08-07 11:10:59,058 ERROR [o.a.s.c.s.SeaTunnel           ] [main] -
===============================================================================



Exception in thread "main" org.apache.seatunnel.core.starter.exception.CommandExecuteException: SeaTunnel job executed failed
	at org.apache.seatunnel.core.starter.seatunnel.command.ClientExecuteCommand.execute(ClientExecuteCommand.java:211)
	at org.apache.seatunnel.core.starter.SeaTunnel.run(SeaTunnel.java:40)
	at org.apache.seatunnel.core.starter.seatunnel.SeaTunnelClient.main(SeaTunnelClient.java:34)
Caused by: org.apache.seatunnel.api.table.factory.FactoryException: ErrorCode:[API-06], ErrorDescription:[Factory initialize failed] - Unable to create a source for identifier 'Hive'.
	at org.apache.seatunnel.api.table.factory.FactoryUtil.createAndPrepareSource(FactoryUtil.java:101)
	at org.apache.seatunnel.engine.core.parse.MultipleTableJobConfigParser.parseSource(MultipleTableJobConfigParser.java:361)
	at org.apache.seatunnel.engine.core.parse.MultipleTableJobConfigParser.parse(MultipleTableJobConfigParser.java:209)
	at org.apache.seatunnel.engine.client.job.ClientJobExecutionEnvironment.getLogicalDag(ClientJobExecutionEnvironment.java:114)
	at org.apache.seatunnel.engine.client.job.ClientJobExecutionEnvironment.execute(ClientJobExecutionEnvironment.java:182)
	at org.apache.seatunnel.core.starter.seatunnel.command.ClientExecuteCommand.execute(ClientExecuteCommand.java:158)
	... 2 more
Caused by: java.lang.NoSuchMethodError: 'void org.apache.hadoop.hive.metastore.HiveMetaStoreClient.<init>(org.apache.hadoop.conf.Configuration)'
	at org.apache.seatunnel.connectors.seatunnel.hive.utils.HiveMetaStoreProxy.<init>(HiveMetaStoreProxy.java:110)
	at org.apache.seatunnel.connectors.seatunnel.hive.utils.HiveMetaStoreProxy.getInstance(HiveMetaStoreProxy.java:139)
	at org.apache.seatunnel.connectors.seatunnel.hive.utils.HiveTableUtils.getTableInfo(HiveTableUtils.java:41)
	at org.apache.seatunnel.connectors.seatunnel.hive.source.config.HiveSourceConfig.<init>(HiveSourceConfig.java:84)
	at org.apache.seatunnel.connectors.seatunnel.hive.source.config.MultipleTableHiveSourceConfig.parseFromLocalFileSourceConfig(MultipleTableHiveSourceConfig.java:52)
	at org.apache.seatunnel.connectors.seatunnel.hive.source.config.MultipleTableHiveSourceConfig.<init>(MultipleTableHiveSourceConfig.java:39)
	at org.apache.seatunnel.connectors.seatunnel.hive.source.HiveSource.<init>(HiveSource.java:43)
	at org.apache.seatunnel.connectors.seatunnel.hive.source.HiveSourceFactory.lambda$createSource$0(HiveSourceFactory.java:46)
	at org.apache.seatunnel.api.table.factory.FactoryUtil.createAndPrepareSource(FactoryUtil.java:113)
	at org.apache.seatunnel.api.table.factory.FactoryUtil.createAndPrepareSource(FactoryUtil.java:74)
	... 7 more

Zeta or Flink or Spark Version

Zeta 2.3.6

Java or Scala Version

java 17

Screenshots

No response

Are you willing to submit PR?

  • [ ] Yes I am willing to submit a PR!

Code of Conduct

matianhe3 avatar Aug 07 '24 03:08 matianhe3