amoro
amoro copied to clipboard
Apache Amoro (incubating) is a Lakehouse management system built on open data lake formats.
Add env preparation without docker.
Fix #108. `ArcticDynamicSource` should implement the `SupportsProjectionPushDown` interface to accelerate data querying. **The brief change log** - `ArcticDynamicSource` and `KafkaDynamicSource` implement the `SupportsProjectionPushDown` interface to support `supportsNestedProjection` and `applyProjection` methods.
If arctic's file-store only have data in base-table, but nothing in change-table, arctic source will reading nothing in ContinuousSplitPlannerImpl#discoverInitialSplits because changeSnapshot in change-table is null. The data in base-table can...
case: Writing CDC data to Arctic tables with primary key via Flink, after multiple update/delete commits,continous optimizing could not be triggered as expected
complete method createInternalRowWriterFactory in ArcticWriter. add some classes for writer. add TestDelegateHiveTable which test for delegate hive table.
Background: There are some requirements for real-time data widening. Now hive supports lookup join, but this solution is not available for production, and the hive table needs to be loaded...
Affected flink versions: 1.12, 1.14. ArcticDynamicSource should implement the `SupportsProjectionPushDown` interface to accelerate data querying.
This issue should affect Flink versions: flink1.12/flink1.14