kafka-connect-fs icon indicating copy to clipboard operation
kafka-connect-fs copied to clipboard

How to connect via FTP

Open FlorianKrippner opened this issue 2 years ago • 1 comments

Sorry for making an issue for this topic, but i am stuck a couple of days already. Maybe someone could help me out :)

I want the 'FileSystem Source Connector' to read a file via ftp and write it to a kafka topic. This is my propertie file right now:

name=FsSourceConnector connector.class=com.github.mmolimar.kafka.connect.fs.FsSourceConnector tasks.max=1 fs.uris=file:///c:/development/kafka/pnw/data/connect-plugin-kafka-connect-fs topic=kafka-connect-fs-test policy.class=com.github.mmolimar.kafka.connect.fs.policy.SimplePolicy policy.recursive=true policy.regexp=^.*.dat$ policy.batch_size=0 policy.cleanup=none file_reader.class=com.github.mmolimar.kafka.connect.fs.file.reader.CobolFileReader file_reader.batch_size=0 file_reader.cobol.copybook.path=file:///c:/development/kafka/pnw/data/connect-plugin-kafka-connect-fs/VEBU.cob file_reader.cobol.reader.is_ebcdic=true file_reader.cobol.reader.is_record_sequence=false

At the moment i have the file for test porpuse locally on my pc and it successfully writing the file to the kafka topic. fs.uris=file:///c:/development/kafka/pnw/data/connect-plugin-kafka-connect-fs

Normally the file is only accessible via ftp. How i can change fs.uris to work with ftp? In the documentation its saying its support ftp, but i couldn't find any more docs about it. image

FlorianKrippner avatar Apr 29 '22 04:04 FlorianKrippner

Hi! Yes, you can access to SFTP or FTP server setting its corresponding URI (ftp://xxxx)

mmolimar avatar May 21 '22 01:05 mmolimar