hudi icon indicating copy to clipboard operation
hudi copied to clipboard

[HUDI-7669] Move config classes and utils to proper places

Open yihua opened this issue 1 year ago • 4 comments

Change Logs

This PR moves config classes and utils to the proper places:

  • Moves ConfigProperty, EnumDescription, EnumFieldDescription, HoodieConfig, TypedProperties from hudi-common to hudi-io module so that classes in hudi-io and hudi-hadoop-common modules can use config-related logic.
  • Moves DFSPropertiesConfiguration class from hudi-common to hudi-hadoop-common module.
  • Moves HoodieNotSupportedException class used by config utils from hudi-common to hudi-io module.
  • Moves basic config utils from ConfigUtils in hudi-common module to ConfigUtils in hudi-io module (tests are moved to TestConfigUtils).
  • Moves Hadoop-related config utils from ConfigUtils to HadoopConfigUtils in hudi-hadoop-common module (tests are moved to TestHadoopConfigUtils).
  • Moves Hudi-specific config utils from ConfigUtils to HoodieConfigUtils in hudi-common module.

Impact

Code refactoring for reuse and better organization.

Risk level

none

Documentation Update

none

Contributor's checklist

  • [ ] Read through contributor's guide
  • [ ] Change Logs and Impact were stated clearly
  • [ ] Adequate tests were added if applicable
  • [ ] CI passed

yihua avatar Apr 25 '24 22:04 yihua

Moves ConfigProperty, EnumDescription, EnumFieldDescription, HoodieConfig, TypedProperties from hudi-common to hudi-io module so that classes in hudi-io and hudi-hadoop-common modules can use config-related logic

It may solves the dependency issue, but from the first sight, these classes does not belong to hudi-io IMO, should we have a separate hudi-config module or hudi-api module (that we can put all the code API or interfaces in hudi core).

danny0405 avatar Apr 25 '24 23:04 danny0405

CI report:

  • 105d20c9eb7387a89552061f32cb346dfdab2bc7 Azure: SUCCESS
Bot commands @hudi-bot supports the following commands:
  • @hudi-bot run azure re-run the last Azure build

hudi-bot avatar Apr 26 '24 00:04 hudi-bot

Moves ConfigProperty, EnumDescription, EnumFieldDescription, HoodieConfig, TypedProperties from hudi-common to hudi-io module so that classes in hudi-io and hudi-hadoop-common modules can use config-related logic

It may solves the dependency issue, but from the first sight, these classes does not belong to hudi-io IMO, should we have a separate hudi-config module or hudi-api module (that we can put all the code API or interfaces in hudi core).

Makes sense. Let me think about this and put up separate PRs.

yihua avatar Apr 26 '24 08:04 yihua

Makes sense. Let me think about this and put up separate PRs.

Sure, please update the description when you finished the change so the reviewer can get a briefing of the changes.

danny0405 avatar Apr 27 '24 01:04 danny0405

Closing this PR as it is no longer required.

yihua avatar May 31 '24 19:05 yihua