huliwuli

Results 9 comments of huliwuli

> Looks like a jackson jar conflict. Is there anything I can do for this issue?

> Finds out where the legacy jackson comes from and remove it from the classpath. Ok, thanks I will try ... since it's on EMR. Not sure whether I have...

> Sorry, Looks like you are using the AWS managed hudi. Can you try using emr-6.15.0 which has hudi 0.14.0 EMR 6.15 worked, I tested it yesterday.

**Regarding Athena Issue:** Due to the small size of parquets, I implemented clustering (inline) with max commits =1 for test. Athena Raises Error: Generic_INTERNAL_ERROR: Can not read value at 0...

> @huliwuli "insert" operation type should handle merging small files. I see you set up a small file size limit of 10 MB. Can you remove that config (default 104857600)...

> @huliwuli Do you see a successful .replacecommit if the clustering was successful. Can you post screenshot for the timeline Thanks for help, I did not see a named "successful.replacecommit"...

> @huliwuli "insert" operation type should handle merging small files. I see you set up small file size limit as 10 MB. can you remove that config (default 104857600) or...

> @huliwuli So, It looks like your per record size is really small. Hudi uses previous commit's statistics to guess future record sizes. For very first commit, it relies on...