datafusion

datafusion copied to clipboard

Support ORDER BY in AggregateUDF

6

### Is your feature request related to a problem or challenge? Some built in aggregates (such as `FIRST_VALUE`, `LAST_VALUE` and `ARRAY_AGG`) support an optional ORDER BY argument that defines the...

alamb

Invalid timezone information is ignored after `Z` in timestamp literals

7

### Describe the bug Some timestamps that should error are actally parsed, leading to confusing cases when users make a mistake For example this timestamp is invalid `'2023-12-05T21:58:10.45ZZTOP'` but it...

alamb

### Describe the bug If you attempt to join two tables on a struct field, the query will plan it successfully, albeit with the struct equality in a the `filter`,...

jacksonrnewhouse

Support more builtin functions provided by spark and missing in datafusion ?

4

### Is your feature request related to a problem or challenge? Comparing functions offered by spark and datafusion, I see that [log1p](https://spark.apache.org/docs/latest/api/sql/index.html#log1p) and [sha1](https://spark.apache.org/docs/latest/api/sql/index.html#sha1) is not supported. There maybe many...

psvri

Implement spilling for PartialSortExec

2

### Is your feature request related to a problem or challenge? `PartialSortExec` was added in https://github.com/apache/arrow-datafusion/issues/7456 / https://github.com/apache/arrow-datafusion/pull/9125 While one of the major benefits of this operator is to reduce...

alamb

doc: RecordBatchReceiverStreamBuilder::spawn_blocking does not abort threads

3

### Describe the bug https://docs.rs/datafusion/latest/datafusion/physical_plan/stream/struct.RecordBatchReceiverStreamBuilder.html#method.spawn_blocking docs say > Spawn a blocking task that will be aborted if this builder (or the stream built from it) are dropped There's no aborting...

tv42

Consolidate interval analysies from `Interval` and `PruningPredicate`

5

### Is your feature request related to a problem or challenge? We now have two ways to do range / interval analysis in DataFusion. Having two representations is challenging because...

alamb

cli: support `~` (tilde) in paths to refer to home directory

2

### Is your feature request related to a problem or challenge? Be able to have `~` in paths in datafusion-cli resolve to user's home directory. (I'm not certain if this...

Jefffrey

Support "A column is known to be entirely NULL" in `PruningPredicate`

11

### Is your feature request related to a problem or challenge? This is broken out from https://github.com/apache/arrow-datafusion/issues/7869 which is describing a slightly different problem [`PruningPredicate`](https://docs.rs/datafusion/latest/datafusion/physical_optimizer/pruning/struct.PruningPredicate.html#) can't be told about columns...

alamb

[Proposal] Ensure consistency between code and documentation.

3

### Is your feature request related to a problem or challenge? There are currently two documentation hosting targets for DataFusion: [docs.rs](https://docs.rs/) (several crates) and [arrow.apache.org](https://arrow.apache.org/datafusion). 1. **docs.rs** - All crates...

ongchi