datafusion
datafusion copied to clipboard
apache
Apache DataFusion SQL Query Engine
## Which issue does this PR close? - Closes #. ## Rationale for this change we have `serialize` already, and it would be nice to have another `deserialize` so we...
## Rationale for this change I wrote this PR mostly to show that it can be done. Replacing `OnceFut` with a `OnceCell` has some advantages though: - It no longer...
## Which issue does this PR close? - Closes #16318. - Relates to #16196 and/or #16301 ## Rationale for this change Yielding to the runtime in Tokio involves unwinding the...
## Which issue does this PR close? - Closes [#16241](https://github.com/apache/datafusion/issues/16241) ## Rationale for this change ## What changes are included in this PR? ## Are these changes tested? ## Are...
I recently gave a talk about using DataFusion to build specialized indexes for querying Parquet: https://www.youtube.com/watch?v=74YsJT1-Rdk I think it would also make a nice blog post on the datafusion site,...
Closes #18824
As discussed in https://github.com/apache/datafusion/pull/16791 the long term plan in my mind (and that I would like to discuss with the community) is to replace `SchemaAdapter` with `PhysicalExprAdapter`. There are multiple...
This is a list of improvements we are working on in ListingTable in DataFusion ### Background DataFusion has a [`ListingTable`](https://github.com/apache/arrow-datafusion/blob/2dad90425bacb98a3c2a4214faad53850c93104e/datafusion/core/src/datasource/listing/table.rs#L443-L508) that effectively reading tables stored in one or more files...
## Which issue does this PR close? - Closes #19286. - Related to #13123 ## Rationale for this change GroupedHashAggregateStream currently always reports that it can spill to the memory...
### Is your feature request related to a problem or challenge? I am currently in need of using datafusion SQL query engine (through delta-rs merge operation) to hash with a...
