hudi icon indicating copy to clipboard operation
hudi copied to clipboard

[HUDI-9424]Support using local timezone when writing flink TIMESTAMP data

Open cmmp6 opened this issue 1 year ago • 8 comments

Change Logs

This PR makes the changes to support using local timezone when writing flink TIMESTAMP data.

Impact

User can use utc or local timezone to write flink TIMESTAMP date.

Risk level (write none, low medium or high below)

low

Documentation Update

Add new flink config option "write.utc-timezone". Default value is "true" for forward compatibility.

Contributor's checklist

  • [x] Read through contributor's guide
  • [x] Change Logs and Impact were stated clearly
  • [x] Adequate tests were added if applicable
  • [ ] CI passed

cmmp6 avatar Jan 31 '24 09:01 cmmp6

PR is to solve problem in https://github.com/apache/hudi/issues/9424

cmmp6 avatar Jan 31 '24 09:01 cmmp6

@danny0405 please review this PR.

cmmp6 avatar Jan 31 '24 09:01 cmmp6

Related items https://github.com/apache/hudi/pull/7886 , https://github.com/apache/hudi/pull/7904

cmmp6 avatar Jan 31 '24 09:01 cmmp6

The travis tests still got falures.

danny0405 avatar Feb 06 '24 03:02 danny0405

The travis tests still got falures.

I find Timezone in JVM Properties is used to convert Flink TimestampData to Timestamp. This is the reason that IT succeeded locally and failed in CI. So I add code to set timezone in order to ensure that IT runs in the same environment.

cmmp6 avatar Feb 17 '24 01:02 cmmp6

@hudi-bot run azure

cmmp6 avatar Feb 19 '24 11:02 cmmp6

There are still test failures: https://dev.azure.com/apache-hudi-ci-org/apache-hudi-ci/_build/results?buildId=22545&view=logs&j=600e7de6-e133-5e69-e615-50ee129b3c08&t=bbbd7bcc-ae73-56b8-887a-cd2d6deaafc7

danny0405 avatar Feb 22 '24 03:02 danny0405

There are still test failures: https://dev.azure.com/apache-hudi-ci-org/apache-hudi-ci/_build/results?buildId=22545&view=logs&j=600e7de6-e133-5e69-e615-50ee129b3c08&t=bbbd7bcc-ae73-56b8-887a-cd2d6deaafc7

Is there any way to rerun failed unit test? The failed unit test succeeded locally and I think the code change is not related with failed unit test. Failed test is a unit test in hudi-spark-client module. image

cmmp6 avatar Feb 27 '24 10:02 cmmp6

@hudi-bot run azure

cmmp6 avatar Mar 01 '24 06:03 cmmp6

CI report:

  • bad9bfef7feb2b29fbdd90c67b2c1fd5e97ff64f Azure: SUCCESS
Bot commands @hudi-bot supports the following commands:
  • @hudi-bot run azure re-run the last Azure build

hudi-bot avatar Mar 01 '24 09:03 hudi-bot