polars icon indicating copy to clipboard operation
polars copied to clipboard

feat(python): add glob flag

Open romanovacca opened this issue 1 year ago • 3 comments

Partially resolves #12876.

It adds a use_glob flag for read_parquet as mentioned in the issue. This issue does NOT address the recursion problem mentioned in the issue.

Do we want to add this logic also for read_csv? If so, should I add it in this PR or make another?

romanovacca avatar Dec 13 '23 20:12 romanovacca

Thank you @romanovacca, but I think we need a bit more here. Most importantly the glob flag should be added at the scan_parquet on the rust side.

ritchie46 avatar Dec 14 '23 07:12 ritchie46

The recursion sees to occur in the if isinstance(source, str) and _is_glob_pattern(source) part, so i also figured that would be a good place to add the use_glob flag. Cant pinpoint yet exactly where the recursion happens. Is this solution more in line with what you had in mind @ritchie46 ?

romanovacca avatar Dec 14 '23 16:12 romanovacca

Not entirely. Where do we use the use_glob flag on the rust side?

ritchie46 avatar Dec 15 '23 09:12 ritchie46

Looking at the changes, a glob flag is added, but it doesn't really seem to be used anywhere?

Since this PR is somewhat outdated, and it's not a full implementation, I'll close it for now. Feel free to finish the implementation and open a new PR!

stinodego avatar Feb 14 '24 19:02 stinodego