facontidavide
detect apache hive directory structures and use them as prefix suggestions
when the parent directories name contains a "=" sign the code traverses upwards until no "=" is found, the parts behind the "=" are joined together with / to create a tree to make the data easy to select when a handling lots of data.
Apache hive format: test ├── year=2021 │ ├── month=1 │ │ ├── file1.parquet │ │ └── file2.parquet │ └── month=2 │ └── file3.parquet
becomes 2021/1/ as the suggestion in the prefix dialog
this is nice, but as a directory like this can contain a LOT of files, I think It would be good to have a popup that display the number and size of the files that will be loaded and potentially a checkbox to select / deselect them
I think, I was not clear enough with my explanation. This does not load all files in the parent directories automatically. It is just changing the suggested prefixes from the filename to the hive structure path of the file