Jon Vexler comments

Results 16 comments of


                                            Jon Vexler

[WIP] [HUDI-6787] Implement the HoodieFileGroupReader API for Hive

@vinothchandar

[WIP] [HUDI-6787] Implement the HoodieFileGroupReader API for Hive

Thanks @bvaradar, we all appreciate it!

[WIP] [HUDI-6787] Implement the HoodieFileGroupReader API for Hive

> Overall looks good to me. @jonvex : What Hive versions are we targeting/testing ? @bvaradar I used the docker demo to test. I think that is using Hive 2....

[HUDI-7045] Create parquet readers inside the reader context and implement schema.on.read in the filegroup reader in spark

Azure CI all passing @yihua

[HUDI-7236] Allow MIT to change partition path when using global index

@hudi-bot run azure

[HUDI-7236] Allow MIT to change partition path when using global index

@nsivabalan The change I made is to use the config from that keyGeneratorWriteConfigOpt when calling mergeIncomingWithExistingRecordWithExpressionPayload. We need to do that because otherwise wrapIntoHoodieRecordPayloadWithParams will wrap it into an expression...

[HUDI-7241] Avoid always broadcast HUDI relation if not using HoodieSparkSessionExtension

Hi @beyond1920 could you please let me know why you closed this? I can assist or take over fixing this. Please let me know if you still think this is...

[HUDI-7244] Ensure HoodieFileGroupReader.close() is called in spark

> > Did we chase down every reader (base files, log files, iterator) part or the new file group and ensured this is the only gap ? @jonvex @linliu-code >...

[HUDI-7244] Ensure HoodieFileGroupReader.close() is called in spark

Azure CI is passsing: https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=21791

[SUPPORT] org.apache.avro.SchemaParseException: Can't redefine: array When there are Top level variables , Struct and Array[struct] (no complex datatype within array[struct])

Ok, so this seems to be the same issue as [https://github.com/apache/hudi/issues/2657](https://github.com/apache/hudi/issues/2657) Here is my runbook that you can use in spark shell. I used Spark 3.1.3 and Hudi 0.12.2 :...