datafusion icon indicating copy to clipboard operation
datafusion copied to clipboard

cli: support `~` (tilde) in paths to refer to home directory

Open Jefffrey opened this issue 1 year ago • 2 comments

Is your feature request related to a problem or challenge?

Be able to have ~ in paths in datafusion-cli resolve to user's home directory.

(I'm not certain if this change can be localized to datafusion-cli only, or would need support in actual datafusion codebase :thinking: )

Describe the solution you'd like

Have both queries succeed from cli:

❯ select count(*) from '/home/jeffrey/Code/arrow-datafusion/benchmarks/data/tpch_sf1/part/part-0.parquet';
+----------+
| COUNT(*) |
+----------+
| 16839    |
+----------+
1 row in set. Query took 0.005 seconds.

❯ select count(*) from '~/Code/arrow-datafusion/benchmarks/data/tpch_sf1/part/part-0.parquet';
Error during planning: table 'datafusion.public.~/Code/arrow-datafusion/benchmarks/data/tpch_sf1/part/part-0.parquet' not found
❯

Describe alternatives you've considered

No response

Additional context

Not sure how this should be handled for Windows?

FYI duckdb supports this (tested on my Linux PC):

D select count(*) from '/home/jeffrey/Code/arrow-datafusion/benchmarks/data/tpch_sf1/part/part-0.parquet';
┌──────────────┐
│ count_star() │
│    int64     │
├──────────────┤
│        16839 │
└──────────────┘
D select count(*) from '~/Code/arrow-datafusion/benchmarks/data/tpch_sf1/part/part-0.parquet';
┌──────────────┐
│ count_star() │
│    int64     │
├──────────────┤
│        16839 │
└──────────────┘
D

Jefffrey avatar Feb 14 '24 10:02 Jefffrey

YES! I have wanted this for a long time.

I think this would be a good first issue too -- it requires figuring out how to enable ~ expansion in paths

alamb avatar Feb 14 '24 14:02 alamb

I can do this one

Lordworms avatar Feb 14 '24 16:02 Lordworms