datafusion-ballista icon indicating copy to clipboard operation
datafusion-ballista copied to clipboard

Support multiple paths for ListingTableScanNode

Open Ted-Jiang opened this issue 3 years ago • 0 comments

Is your feature request related to a problem or challenge? Please describe what you are trying to do. After support https://github.com/apache/arrow-datafusion/pull/2775 We could support multiple paths in ListingTableScanNode for ballista.

Describe the solution you'd like update datafusion in ballista and change

message ListingTableScanNode {
  string table_name = 1;
  string path = 2;
  string file_extension = 3;

to

message ListingTableScanNode {
  string table_name = 1;
  repeated string path = 2;
  string file_extension = 3;
  ProjectionColumns projection = 4;

Describe alternatives you've considered A clear and concise description of any alternative solutions or features you've considered.

Additional context Add any other context or screenshots about the feature request here.

Ted-Jiang avatar Jun 25 '22 13:06 Ted-Jiang