seatunnel icon indicating copy to clipboard operation
seatunnel copied to clipboard

[Bug] [file] Plugin PluginIdentifier{engineType='flink', pluginType='source', pluginName='file'} not found in flink running

Open Alxe1 opened this issue 2 years ago • 2 comments

Search before asking

  • [X] I had searched in the issues and found no similar issues.

What happened

The program finished with the following exception:

org.apache.flink.client.program.ProgramInvocationException: The main method caused an error: Plugin PluginIdentifier{engineType='flink', pluginType='source', pluginName='file'} not found.

SeaTunnel Version

2.3.0-beta

SeaTunnel Config

env {

}

source {
  file {
    format.type="csv"
    path="hdfs://master:8020/my_file.csv"
	result_table_name="local_table"
	schema="[{\"type\": \"string\"}, {\"type\": \"string\"}]"
  }
}

transform {
}

sink {

  Console {}

}

Running Command

./bin/start-seatunnel-flink.sh -c test.conf

Error Exception

------------------------------------------------------------
 The program finished with the following exception:

org.apache.flink.client.program.ProgramInvocationException: The main method caused an error: Plugin PluginIdentifier{engineType='flink', pluginType='source', pluginName='file'} not found.
	at org.apache.flink.client.program.PackagedProgram.callMainMethod(PackagedProgram.java:372)
	at org.apache.flink.client.program.PackagedProgram.invokeInteractiveModeForExecution(PackagedProgram.java:222)
	at org.apache.flink.client.ClientUtils.executeProgram(ClientUtils.java:114)
	at org.apache.flink.client.cli.CliFrontend.executeProgram(CliFrontend.java:812)
	at org.apache.flink.client.cli.CliFrontend.run(CliFrontend.java:246)
	at org.apache.flink.client.cli.CliFrontend.parseAndRun(CliFrontend.java:1054)
	at org.apache.flink.client.cli.CliFrontend.lambda$main$10(CliFrontend.java:1132)
	at java.security.AccessController.doPrivileged(Native Method)
	at javax.security.auth.Subject.doAs(Subject.java:422)
	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1926)
	at org.apache.flink.runtime.security.contexts.HadoopSecurityContext.runSecured(HadoopSecurityContext.java:41)
	at org.apache.flink.client.cli.CliFrontend.main(CliFrontend.java:1132)
Caused by: java.lang.RuntimeException: Plugin PluginIdentifier{engineType='flink', pluginType='source', pluginName='file'} not found.
	at org.apache.seatunnel.plugin.discovery.AbstractPluginDiscovery.createPluginInstance(AbstractPluginDiscovery.java:137)
	at org.apache.seatunnel.plugin.discovery.AbstractPluginDiscovery.createPluginInstance(AbstractPluginDiscovery.java:99)
	at org.apache.seatunnel.core.flink.config.FlinkExecutionContext.lambda$getSources$0(FlinkExecutionContext.java:68)
	at java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:193)
	at java.util.ArrayList$ArrayListSpliterator.forEachRemaining(ArrayList.java:1384)
	at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:482)
	at java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:472)
	at java.util.stream.ReduceOps$ReduceOp.evaluateSequential(ReduceOps.java:708)
	at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
	at java.util.stream.ReferencePipeline.collect(ReferencePipeline.java:566)
	at org.apache.seatunnel.core.flink.config.FlinkExecutionContext.getSources(FlinkExecutionContext.java:71)
	at org.apache.seatunnel.core.flink.command.FlinkApiTaskExecuteCommand.execute(FlinkApiTaskExecuteCommand.java:69)
	at org.apache.seatunnel.core.base.Seatunnel.run(Seatunnel.java:39)
	at org.apache.seatunnel.core.flink.SeatunnelFlink.main(SeatunnelFlink.java:34)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at org.apache.flink.client.program.PackagedProgram.callMainMethod(PackagedProgram.java:355)
	... 11 more

Flink or Spark Version

flink 1.13.6

Java or Scala Version

java8

Screenshots

No response

Are you willing to submit PR?

  • [ ] Yes I am willing to submit a PR!

Code of Conduct

Alxe1 avatar Nov 23 '22 10:11 Alxe1

you need to execute the following command to install the connector sh bin/install_plugin.sh 2.3.3

FWLamb avatar Nov 23 '22 10:11 FWLamb

you need to execute the following command to install the connector sh bin/install_plugin.sh 2.3.0-beta

Yes, I have installed all the connectors.

--flink-connectors--
seatunnel-connector-flink-assert
seatunnel-connector-flink-druid
seatunnel-connector-flink-fake
seatunnel-connector-flink-file
seatunnel-connector-flink-influxdb
seatunnel-connector-flink-jdbc
seatunnel-connector-flink-kafka
seatunnel-connector-flink-socket
seatunnel-connector-flink-http
seatunnel-connector-flink-clickhouse
seatunnel-connector-flink-console
seatunnel-connector-flink-doris
seatunnel-connector-flink-elasticsearch7

Alxe1 avatar Nov 24 '22 01:11 Alxe1

This issue has been automatically marked as stale because it has not had recent activity for 30 days. It will be closed in next 7 days if no further activity occurs.

github-actions[bot] avatar Dec 26 '22 00:12 github-actions[bot]

This issue has been closed because it has not received response for too long time. You could reopen it if you encountered similar problems in the future.

github-actions[bot] avatar Jan 03 '23 00:01 github-actions[bot]