hudi
hudi copied to clipboard
apache
[HUDI-7989] Fix secondary index updates
Change Logs
Secondary index update fails because the current logic created a wasteful secondary_index_ partition in addition to the actual index partition at the time of initialization, and was unable to detect the index partition correctly. This PR fixes the problem:
- Removes unnecessary
secondary_index_partition - Fixes the MDT writer methods to work with exact MDT partition name instead of
MetadataPartitionType. - Added a test to udpate secondary keys
Impact
fix index update
Risk level (write none, low medium or high below)
low
Documentation Update
Describe any necessary documentation update if there is any new feature, config, or user-facing change. If not, put "none".
- The config description must be updated if new configs are added or the default value of the configs are changed
- Any new feature or user-facing change requires updating the Hudi website. Please create a Jira ticket, attach the ticket number here and follow the instruction to make changes to the website.
Contributor's checklist
- [ ] Read through contributor's guide
- [ ] Change Logs and Impact were stated clearly
- [ ] Adequate tests were added if applicable
- [ ] CI passed
index on a column that is part of record key where in record key is complex. for eg, say record key is on colA and colB. can we add sec or functional index on colA.
Yes, sec index can be created. The only condition for sec index to create is that the column should exist and record index should exist. There is no check as such whether other index types are enabled for same column or not.
