khiops icon indicating copy to clipboard operation
khiops copied to clipboard

Support file globbing for "any" filesystem (thus for local one too)

Open tramora opened this issue 5 months ago • 0 comments

Description

Currently in khiops (versions 10 and 11), the file globbing is supported for remote file systems (S3, GCS) via the corresponding drivers.

Example of a path with a globbing pattern :

gs://mybucket/folder/*/*.csv

All the files are concatenated to produce one source of data.

Questions/Ideas

An internal customer finds strange that this globbing feature is not supported for example for the local system too.

For his CI/CD chain and his production system he would like to be able to configure as easily as below

  • CI/CD (Automated Integration Test) :
local path : 
/folder/*/*.csv
  • Q/A, Production :
remote path : 
gs:/bucket/folder/*/*.csv

If it does not lead to a major rewrite it would be nice to have this homogenous globbing feature for every filesystem.

tramora avatar Jul 24 '25 10:07 tramora