gh-ost icon indicating copy to clipboard operation
gh-ost copied to clipboard

fix data-integrity: timestamp convert to datetime scenario

Open cenkore opened this issue 3 years ago • 3 comments

related: #161 #247 #269 ..

This PR is to fix the data integrity problem in timestamp conversion datetime scenarios.

Env

Variable_name Value
system_time_zone CST
time_zone +08:00

Reason The timestamp time_zone parsed from binlog is CST +08:00, go-sql-driver will convert it to UTC +00:00 by default, and finally when applying to the database, even though add session query set @@session.time_zone='+00:00', but because the column type has been changed to datetime, the time zone did not do conversion again, so the time zone was actually only converted by half, causing the data written to _xx_gho is 8 hours smaller.

Solution In this case, change the time zone from +00:00 to the local system time zone, which is the opposite of datetime convert to timestamp.

Thank you !

cenkore avatar May 08 '21 05:05 cenkore

Fixed merge conflict for testing

dm-2 avatar Jan 04 '22 14:01 dm-2

@cenkore / @dm-2 / @rashiq: as this converts data I'd like to be very sure it works now and in the future

Are there any tests that we can add to the localtests/ dir that would:

  1. Confirm master currently has a problem (test fails before changes in this PR)
  2. Confirm the logic in this PR resolves the problem? 🤔

timvaillancourt avatar Jul 06 '22 22:07 timvaillancourt

@cenkore see my previous comment 🙏

@timvaillancourt Sorry for taking a long time to reply you. PTAL.

cenkore avatar Jul 15 '22 06:07 cenkore