dask-sql
dask-sql copied to clipboard
dask-contrib
Distributed SQL Engine in Python using Dask
**Is your feature request related to a problem? Please describe.** Testing, tweaking, repeat needs to happen for the Rex (Row Expression) logic after all of the porting is complete. Rex...
**Is your feature request related to a problem? Please describe.** As a work around to a Datafusion limitation a private member, `table_name` was introduced to the `DaskTableProvider` in `table.rs`. This...
**Is your feature request related to a problem? Please describe.** Apache Datafusion cannot currently parse the `Filter` syntax of a selective aggregate query. The current implementation with Apache Calcite does...
Prediction with empty partitions fails on sklearn dask-ml Models . This is because sklearn currently errors on empty frames. I am opening this issue here to track the best approach...
**What is your question?** _TLDR - there's some costly null handing in our join code to maintain SQL compatibility - should we remove this like we did with groupby null...
``` from dask_sql import Context import pandas as pd import dask.dataframe as dd c = Context() pd.DataFrame({'id': [0, 1, 2]}).to_parquet('/data/test/part.0.parquet') # this works c.sql(""" CREATE OR REPLACE TABLE test WITH...
Recently, there's been some thought into what dependencies Dask-SQL should be packaged with or not - there are a lot of packages included included in the [standard conda package](https://github.com/dask-contrib/dask-sql/blob/c3ad6a9f6b01ce02127fde7501eaf322c8160f7e/setup.py#L89-L100) that...
I'm trying to use a `CREATE TABLE WITH (... filters=[...])` on a Parquet dataset, and trying to achieve row group filtering based on filters supplied in the `CREATE TABLE` statement,...
**Is your feature request related to a problem? Please describe.** Dask supports optimizing joins when certain columns are indexed or their divisions are known. It might make sense to see...
Closes #391 This PR switches around some of the `logger` / `warnings` in an effort to make warnings less intrusive to users, while also removing some deprecation code that is...