iceberg icon indicating copy to clipboard operation
iceberg copied to clipboard

Parquet: Set parquet bloom filter config with compatible column name

Open ConeyLiu opened this issue 3 years ago • 2 comments

This PR improves the config setting for the parquet bloom filter.

  1. Log warn for those don't exist column or the column type is unsupported (boolean or complex data type).
  2. Convert the column name to a compatible one. This is because we make the column name compatible if it contains special character when we write the parquet file.

ConeyLiu avatar Aug 04 '22 09:08 ConeyLiu

Hi, @huaxingao @rdblue @kbendick . Could you help to review this when you are free? Thanks a lot.

ConeyLiu avatar Aug 04 '22 09:08 ConeyLiu

Thanks @rdblue @kbendick @zhongyujiang @huaxingao for your time to review. Comments have been addressed. Pls take another look when you are free.

ConeyLiu avatar Aug 08 '22 04:08 ConeyLiu

@huaxingao can you take another look and push forward ? thanks

zinking avatar Mar 27 '23 11:03 zinking

LGTM

huaxingao avatar Mar 27 '23 14:03 huaxingao

cc @rdblue Could you please take another look when you have a moment?

huaxingao avatar Mar 27 '23 14:03 huaxingao