hudi
hudi copied to clipboard
[HUDI-3478] Implement CDC Read in Spark
Change Logs
This PR is going to support CDC Read in Spark.
The changes are listed:
- extract some cdc common classes in
hudi-commonthat will be used to implement CDCReader for different engines; CDCReaderto response the CDC query on spark.
Impact
Low
@xushiyan @prasannarajaperumal @alexeykudinkin please continue to review this. and i will submit a following commit that use the avro format, not json format, to persist the fields.
cancelling all azure CI runs for now to investigate CI flakiness. will retrigger build once we are in stable state. sorry about the inconvenience.
CI report:
- e2989860bc2240b23ba1a7cfc1efad59c5d5ead6 Azure: SUCCESS
Bot commands
@hudi-bot supports the following commands:@hudi-bot run azurere-run the last Azure build