Xianyang Liu comments

Results 62 comments of


                                            Xianyang Liu

Core: Add schema_id to ContentFile/ManifestFile

Thanks @szehon-ho @aokolnychyi > Will this mean all evaluator logic will have to change to be schema specific? Is there a simple example how this will be consumed? We don't...

Core: Add schema_id to ContentFile/ManifestFile

Hi @rdblue @szehon-ho @aokolnychyi do you have any time to look at this again?

Core: Add schema_id to ContentFile/ManifestFile

Hi @rdblue @szehon-ho @aokolnychyi @RussellSpitzer @nastra, could you help to review this? This should be useful for tables with frequent column additions or deletions.

DecoderResolver may lead to OOM of flink jobs writing to iceberg tables

We have met a similar problem and I have submitted a PR for this. @stevenzwu Could you help to review it?

[WIP][Proposal] PARQUET-2430: Add parquet joiner

The idea looks good to me too, actually, we have similar requirements to update history data. Feel free to ping me when this is ready for review.

[WIP][Proposal] PARQUET-2430: Add parquet joiner

Thanks @MaxNevermind for the great work and thanks @wgtmac for pinging me. I am sorry for the late reply.

PARQUET-2415: Reuse hadoop file status and footer in ParquetRecordReader

Thanks @wankunde for the contribution. And thanks @wgtmac for ping me.

PARQUET-1381: Support merging of rowgroups during file rewrite

Hi @MaheshGPai, thanks for the contribution. If you don't have time to work on this, I can continue with it.

PARQUET-1381: Support merging of rowgroups during file rewrite

OK, I will deep into it.

[Parquet] Eagerly fetch row groups when reading parquet

Should we use some parameters to enable this or not? Because prefetch row groups meaning need more memory. @singhpk234 is there any benchmark result to show?