[Bug] [Ftp source] java.lang.NoClassDefFoundError: org/apache/hadoop/fs/FSDataInputStream

[bitian]

Search before asking

• [X] I had searched in the issues and found no similar issues.

What happened

when I use ftp connect,there wa a error:

SeaTunnel Version

2.3.0-beat

SeaTunnel Config

config/ftp.conf :
env {
  execution.parallelism = 1
  job.mode = "BATCH"
}

source {
  FtpFile {
    path = "/var/ftp/pub/1.txt"
    host = "192.168.83.180"
    port = 21
    user = test
    password = test
    type = "text"
    schema = {
      name = string
      age = int
    }
    delimiter = "#"
  }
}

transform {
}

sink {
  Console {}
}

Running Command

run command:
./bin/seatunnel.sh --config config/ftp.conf

Error Exception

Exception in thread "main" java.lang.NoClassDefFoundError: org/apache/hadoop/fs/FSDataInputStream
	at org.apache.seatunnel.connectors.seatunnel.file.config.FileFormat$2.getReadStrategy(FileFormat.java:55)
	at org.apache.seatunnel.connectors.seatunnel.file.source.reader.ReadStrategyFactory.of(ReadStrategyFactory.java:32)
	at org.apache.seatunnel.connectors.seatunnel.file.ftp.source.FtpFileSource.prepare(FtpFileSource.java:60)
	at org.apache.seatunnel.engine.core.parse.ConnectorInstanceLoader.loadSourceInstance(ConnectorInstanceLoader.java:60)
	at org.apache.seatunnel.engine.core.parse.JobConfigParser.sampleAnalyze(JobConfigParser.java:291)
	at org.apache.seatunnel.engine.core.parse.JobConfigParser.parse(JobConfigParser.java:119)
	at org.apache.seatunnel.engine.client.job.JobExecutionEnvironment.getLogicalDag(JobExecutionEnvironment.java:135)
	at org.apache.seatunnel.engine.client.job.JobExecutionEnvironment.execute(JobExecutionEnvironment.java:127)
	at org.apache.seatunnel.core.starter.seatunnel.command.ClientExecuteCommand.execute(ClientExecuteCommand.java:71)
	at org.apache.seatunnel.core.starter.Seatunnel.run(Seatunnel.java:39)
	at org.apache.seatunnel.core.starter.seatunnel.SeaTunnelClient.main(SeaTunnelClient.java:31)
Caused by: java.lang.ClassNotFoundException: org.apache.hadoop.fs.FSDataInputStream
	at java.net.URLClassLoader.findClass(URLClassLoader.java:387)
	at java.lang.ClassLoader.loadClass(ClassLoader.java:418)
	at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:352)
	at java.lang.ClassLoader.loadClass(ClassLoader.java:351)
	... 11 more

Flink or Spark Version

No response

Java or Scala Version

No response

Screenshots

No response

Are you willing to submit PR?

• [ ] Yes I am willing to submit a PR!

Code of Conduct

• [X] I agree to follow this project's Code of Conduct

[TyrantLucifer]

Do you add hadoop dependencies in your classpath?

[bitian]

Do you add hadoop dependencies in your classpath?

I can't add it,how can I do it?Do I need to install Hadoop?

[TyrantLucifer]

You need add hadoop dependency jars such as hadoop-common in classpath of ST.

[bitian]

You need add hadoop dependency jars such as hadoop-common in classpath of ST.

Thank you ,I have add dependencyies in plugins folder of ST,include hadoop-common-2.6.5.jar etc.

but now the following error occurred:

Is it a dependency version problem or missing other dependencies?

[lianghuan-xatu]

You need add hadoop dependency jars such as hadoop-common in classpath of ST.

Thank you ,I have add dependencyies in plugins folder of ST,include hadoop-common-2.6.5.jar etc.

but now the following error occurred:

Is it a dependency version problem or missing other dependencies?

You can try to add hadoop-hdfs dependency to see if this problem can be solved.

[xiehanghang]

You need add hadoop dependency jars such as hadoop-common in classpath of ST.

Thank you ,I have add dependencyies in plugins folder of ST,include hadoop-common-2.6.5.jar etc. but now the following error occurred: Is it a dependency version problem or missing other dependencies?

You can try to add hadoop-hdfs dependency to see if this problem can be solved.

You need add hadoop dependency jars such as hadoop-common in classpath of ST.

Thank you ,I have add dependencyies in plugins folder of ST,include hadoop-common-2.6.5.jar etc.

but now the following error occurred:

Is it a dependency version problem or missing other dependencies?

我遇到了跟你一样的问题，请问下你后来如何解决了

[kevinjmh]

Dear all, you can download hadoop shade jar from mvnrepository and then put it into folder ${Seatunnel_HOME}/lib