amoro icon indicating copy to clipboard operation
amoro copied to clipboard

[Feature]: Improve Mixed format

Open zhoujinsong opened this issue 1 year ago • 0 comments

Description

Improve the compatibility of the Mixed Format with Iceberg to reduce the user's usage burden.

Use case/motivation

The goal of the mixed format is to enhance the functionality and performance of the Iceberg format in streaming scenarios. In order to support MOR (Merge-On-Read) on mixed formats, the Amoro community provides packages for Spark and Trino, albeit at the cost of increased user burden and community maintenance.

The subsequent focus for the mixed format should return to enhancing its capabilities in streaming scenarios while maintaining compatibility with the Iceberg format in offline and analytical contexts. Specifically, in engines like Spark and Trino, there should be a greater emphasis on directly utilizing the Iceberg connector to access the BaseStore.

Describe the solution

No response

Subtasks

No response

Related issues

No response

Are you willing to submit a PR?

  • [ ] Yes I am willing to submit a PR!

Code of Conduct

zhoujinsong avatar Feb 05 '24 08:02 zhoujinsong