ibis
ibis copied to clipboard
feat: support `read_parquet`/`read_csv` for most backends
Currently we only support read_parquet for backends that have native support (like duckdb). In contrast, we support to_parquet for all backends, falling back to a common pyarrow implementation if a backend doesn't natively support it.
To provide more uniform feature coverage, we could write an equivalent common pyarrow (or other) implementation of our IO input methods (read_parquet/read_csv/...) that backends like postgres could fall back on.
If duckdb is installed, there's also the option of using duckdb for faster support for parquet/csv table loading. I think that could be done in a 2nd pass, see #8110, but a pass using pyarrow first to cover all backends makes sense to me as a good first step.
HI @jcrist I could take this one. Will do it one by one.
Plan to add read_parquet and read_csv. How about read_json and read_delta?
Any updates on this one?