Consider adding DataFusion (part of Apache Arrow)

[andygrove]

Hello,

Would it be possible to add DataFusion to the database?

DataFusion is an embedded in-memory query engine (not a full RDBMS) that uses the Apache Arrow memory model and supports SQL and DataFrame APIs for OLAP and ETL workloads. It supports object stores (S3, GCS, Azure Blob Storage) and CSV, Parquet, JSON, and Avro sources.

One of the unique features is its extensibility in many areas (parser, planner, optimizer, execution, data sources, etc.) and it is becoming widely used as the foundation for other databases engines, such as IOx from InfluxData. It is inspired by Apache Calcite and Apache Spark and aims for Postgres compatibility.

The source repo is at https://github.com/apache/arrow-datafusion/.

Thanks for your consideration.