amoro
amoro copied to clipboard
[ARCTIC-108][Flink] Support projection push down to arctic source reader
Fix #108.
ArcticDynamicSource should implement the SupportsProjectionPushDown interface to accelerate data querying.
The brief change log
ArcticDynamicSourceandKafkaDynamicSourceimplement theSupportsProjectionPushDowninterface to supportsupportsNestedProjectionandapplyProjectionmethods.
@SteNicholas Thanks for your work.
LogDataJsonDeserialization will deserialize the byte[] data to RowData by Iceberg's Schema.
You can find related code here.
I think we should also change the schema when the projection is pushed down.
What do you think?
@YesOrNo828, thanks for your suggestion. I have addressed the above comments and updated the schema in LogDynamicSource for deserialization. PTAL.