datafusion icon indicating copy to clipboard operation
datafusion copied to clipboard

General ticket for Array/List data type

Open izveigor opened this issue 2 years ago • 2 comments

Is your feature request related to a problem or challenge?

This is a common ticket for all other tickets. The ticket was created for the next purposes:

  • Track the progress of implementing array functions
  • Group issues by category
  • Helps developers better understand problem selection

Describe the solution you'd like

Tickets:

Array creating:

  • https://github.com/apache/arrow-datafusion/issues/6808
  • https://github.com/apache/arrow-datafusion/issues/6556
  • https://github.com/apache/arrow-datafusion/issues/6561
  • https://github.com/apache/arrow-datafusion/issues/6887
  • https://github.com/apache/arrow-datafusion/issues/6993

Array functions:

  • https://github.com/apache/arrow-datafusion/issues/6779
  • https://github.com/apache/arrow-datafusion/issues/6560
  • https://github.com/apache/arrow-datafusion/issues/6555
  • https://github.com/apache/arrow-datafusion/issues/6804
  • https://github.com/apache/arrow-datafusion/issues/6557
  • https://github.com/apache/arrow-datafusion/issues/6119
  • https://github.com/apache/arrow-datafusion/issues/6075
  • https://github.com/apache/arrow-datafusion/issues/6849
  • https://github.com/apache/arrow-datafusion/issues/6972
  • https://github.com/apache/arrow-datafusion/issues/6992

Array operators:

  • https://github.com/apache/arrow-datafusion/issues/6806
  • https://github.com/apache/arrow-datafusion/issues/6603
  • https://github.com/apache/arrow-datafusion/issues/6884

Cast:

  • https://github.com/apache/arrow-datafusion/issues/6558

Concept:

  • https://github.com/apache/arrow-datafusion/issues/6980
  • https://github.com/apache/arrow-datafusion/issues/6973
  • https://github.com/apache/arrow-datafusion/issues/6974
  • https://github.com/apache/arrow-datafusion/issues/6975
  • https://github.com/apache/arrow-datafusion/issues/6976
  • https://github.com/apache/arrow-datafusion/issues/6977
  • https://github.com/apache/arrow-datafusion/issues/6978
  • https://github.com/apache/arrow-datafusion/issues/6979
  • https://github.com/apache/arrow-datafusion/issues/6981

Other:

  • https://github.com/apache/arrow-datafusion/issues/6985
  • https://github.com/apache/arrow-datafusion/issues/6602
  • https://github.com/apache/arrow-datafusion/issues/6559
  • https://github.com/apache/arrow-datafusion/issues/6961
  • https://github.com/apache/arrow-datafusion/issues/7146
  • https://github.com/apache/arrow-datafusion/issues/7147

Discussions:

  • https://github.com/apache/arrow-datafusion/discussions/6855

Describe alternatives you've considered

Additional context

  • https://github.com/apache/arrow-datafusion/issues/6980
  • https://github.com/apache/arrow-datafusion/issues/2326
  • https://github.com/apache/arrow-datafusion/issues/6780

izveigor avatar Jul 06 '23 10:07 izveigor

Any idea whether aggregations on array elements are covered in this epic? I had no success so far:

> select max(unnest([1,3,5]));
type_coercion
caused by
Internal error: Unnest should be rewritten to LogicalPlan::Unnest before type coercion.
This was likely caused by a bug in DataFusion's code and we would welcome that you file an bug report in our issue tracker

ahirner avatar May 03 '24 03:05 ahirner

Yes! But since we are working on strengthening existing function, so not yet works on these.

On Fri, May 3, 2024, 11:50 AM Alexander Hirner @.***> wrote:

Any idea whether aggregations on array elements are covered in this epic? I had no success so far:

select max(unnest([1,3,5])); type_coercion caused by Internal error: Unnest should be rewritten to LogicalPlan::Unnest before type coercion. This was likely caused by a bug in DataFusion's code and we would welcome that you file an bug report in our issue tracker

— Reply to this email directly, view it on GitHub https://github.com/apache/datafusion/issues/6863#issuecomment-2092073806, or unsubscribe https://github.com/notifications/unsubscribe-auth/ADZCLR4DOSM7XTQZL2F5CDLZAMCQ7AVCNFSM6AAAAAA2AGNMUKVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDAOJSGA3TGOBQGY . You are receiving this because you are subscribed to this thread.Message ID: @.***>

jayzhan211 avatar May 03 '24 10:05 jayzhan211