datafusion
datafusion copied to clipboard
apache
Apache DataFusion SQL Query Engine
### Is your feature request related to a problem or challenge? https://github.com/apache/datafusion/pull/12863 updated DataFusion with changes from Substrait 0.58.0 The Substrait changes included https://github.com/substrait-io/substrait/pull/706, which added a new `grouping_expressions` field...
### Describe the bug I am attempting to optimize an inner join on two hive-partitioned parquet datasets `lhs` and `rhs`, joined on one column `join_col`. In a base working query,...
### Is your feature request related to a problem or challenge? _No response_ ### Describe the solution you'd like Applies an offset to the unixtime after the cast. ### Describe...
### Is your feature request related to a problem or challenge? It would be helpful to have spilling support for the HashJoin. If there is not enough memory on the...
This is a test to try and help ensure object store 0.11.1 has no breaking API changes Pin to pre-release https://github.com/apache/arrow-rs/pull/6566 (this PR should not be merged)
### Is your feature request related to a problem or challenge? We have had good luck writing up quarterly updates for DataFusion, most recently: https://datafusion.apache.org/blog/2024/07/24/datafusion-40.0.0/ See https://github.com/apache/datafusion/issues/9602 ### Describe the...
Draft until we have completely migrated all function documentation. The CI will fail until all currently existing functions are documented ## Which issue does this PR close? Closes https://github.com/apache/datafusion/issues/12872 ##...
In https://github.com/apache/datafusion/pull/12881, https://github.com/apache/datafusion/pull/12890 it turned out that `make_function_scalar_inputs_return_type` may lead to less performant code. In https://github.com/apache/datafusion/pull/12909 it turned out that `make_function_inputs2` may lead to less performant code. Revise remaining usages...
### Describe the bug a filter should not be pushed down through a distinct on because it will change the results. The PushDownFilter optimizer rule does not behave correctly. ###...
Follow on to https://github.com/apache/arrow-datafusion/issues/9506 The idea is to support arbitrary expressions that can be consolidated to a constant in the LIMIT clause. For example ```sql ❯ select * from (values...