bitsail icon indicating copy to clipboard operation
bitsail copied to clipboard

[Improve][Connector] Change `path_list` option setting using concating string with comma `,` to string list for FtpConnector

Open Jake-00 opened this issue 2 years ago • 7 comments
trafficstars

Description

For FtpConnector, currently we can specify directories to read by setting configuration json file using option path_list, but it can't specify several files' path to read.

Suggetion:

  1. currently path_list specify a string contains path list, it is better to change to specify a string list like below
  2. update documentation of FtpCoinnector
"reader": {
      "path_list": ["/data/json/upload1/test1.json", "/data/json/upload2/"]
}

BitSail Component or Code Module

BitSail Connector

Are you willing to submit PR?

  • [ ] Yes, I am willing to submit a PR!

Code of Conduct

Jake-00 avatar Feb 24 '23 16:02 Jake-00

This issue maybe good for beginner.

Jake-00 avatar Feb 25 '23 09:02 Jake-00

@Jake-00 Thanks for your issue, you suggestion is we just not support directories right, we can use file path directly?

hk-lrzy avatar Feb 28 '23 06:02 hk-lrzy

@hk-lrzy To read directories already support, but to read multiple specified files do not support yet.

Jake-00 avatar Feb 28 '23 06:02 Jake-00

@hk-lrzy To read directories already support, but to read multiple specified files do not support yet.

I see, can you use same parameter to support? like we check the path is file or directory, if directory we support to list it and if it is file we support read it directly.

I don't want to add parameter because user need to know use which one, it's not necessary for user.

hk-lrzy avatar Mar 02 '23 12:03 hk-lrzy

Sounds more reasonable, shall I change the issue description?

Jake-00 avatar Mar 02 '23 13:03 Jake-00

Sounds more reasonable, shall I change the issue description?

Sure~

hk-lrzy avatar Mar 03 '23 02:03 hk-lrzy

When diving deeper in FtpConnector, it already supports to read specified files and directories. Issue description is wrong and I change the description of this issue. Could we add tag good-first-issue?

Jake-00 avatar Mar 19 '23 03:03 Jake-00