huliwuli
huliwuli
> Looks like a jackson jar conflict. Is there anything I can do for this issue?
> Finds out where the legacy jackson comes from and remove it from the classpath. Ok, thanks I will try ... since it's on EMR. Not sure whether I have...
> Sorry, Looks like you are using the AWS managed hudi. Can you try using emr-6.15.0 which has hudi 0.14.0 EMR 6.15 worked, I tested it yesterday.
@ad1happy2go Thanks for the reply. I used insert operation.
**Regarding Athena Issue:** Due to the small size of parquets, I implemented clustering (inline) with max commits =1 for test. Athena Raises Error: Generic_INTERNAL_ERROR: Can not read value at 0...
> @huliwuli "insert" operation type should handle merging small files. I see you set up a small file size limit of 10 MB. Can you remove that config (default 104857600)...
> @huliwuli Do you see a successful .replacecommit if the clustering was successful. Can you post screenshot for the timeline Thanks for help, I did not see a named "successful.replacecommit"...
> @huliwuli "insert" operation type should handle merging small files. I see you set up small file size limit as 10 MB. can you remove that config (default 104857600) or...
> @huliwuli So, It looks like your per record size is really small. Hudi uses previous commit's statistics to guess future record sizes. For very first commit, it relies on...