datafusion

datafusion copied to clipboard

Add an example of embedding indexes *inside* a parquet file

7

### Is your feature request related to a problem or challenge? One of the common criticisms of parquet based query systems is that they don't have some particular type of...

alamb

RightMark Join support to SortMergeJoin execution

1

@jonathanc-n did a great job adding a RightMark Join, it would be needed to add the same support for Sort Merge Join It is needed to add RightMark Join support...

comphead

POC: Reduce `Arc` cloning on hashmap build side

2

## Which issue does this PR close? - Closes #. ## Rationale for this change ## What changes are included in this PR? ## Are these changes tested? ## Are...

jonathanc-n

Partly closes #16303 The purpose of this PR is to enable using CREATE command with glob pattern and a URL scheme - i.e., ``` CREATE EXTERNAL TABLE ee3 STORED AS...

a-agmon

Enable merge queue in github to avoid commit confliction.

10

### Describe the bug See https://github.com/apache/arrow-datafusion/pull/6875 for details. We can enable merge queue to avoid such kind of conflicts. You can find out details here: https://docs.github.com/en/repositories/configuring-branches-and-merges-in-your-repository/configuring-pull-request-merges/managing-a-merge-queue ### To Reproduce _No...

liurenjie1024

Upgrade to hashbrown 0.15.1: migrate from `hashbrown::raw::RawTable` to `hashbrown::hash_table::HashTable`

9

# What Migrate from `hashbrown::raw::RawTable` to `hashbrown::hash_table::HashTable`. # Why `RawTable` and `raw_entry` are removed in hashbrown 0.15.1. See https://github.com/apache/datafusion/pull/13256 . Personally I think it's the right thing to do because...

crepererum

## Which issue does this PR close? - Closes #. ## Rationale for this change ## What changes are included in this PR? ## Are these changes tested? ## Are...

xudong963

Investigate performance tradeoff in compressing spill files

2

### Is your feature request related to a problem or challenge? Part of https://github.com/apache/datafusion/issues/16065 , Related to https://github.com/apache/datafusion/issues/14078 ### Background PR https://github.com/apache/datafusion/pull/16268 will introduce compression option when writing spill files...

ding-young

feat: implement GroupsAccumulator for `count(DISTINCT)` aggr

12

Signed-off-by: Ruihang Xia ## Which issue does this PR close? Related to #5472 ## Rationale for this change Implement group accumulator for distinct count aggr fn. In `hits.parquet` dataset from...

waynexia

Draft: Use take-in kernel in repartitioning

5

Combined with https://github.com/apache/arrow-rs/pull/7325, tries to use the take_in kernel in repartitioning. The goal is to elide the coalesce step after repartitioning.

ctsk