gh-ost icon indicating copy to clipboard operation
gh-ost copied to clipboard

About adding unique index lost data - There are delete operation scenarios

Open suyestyle opened this issue 3 years ago • 3 comments

Is it possible to solve this scenario?

Add two calculators when there is a delete operation.

  1. The deletion times of the original table are recorded in delete1

  2. The deletion times of the new table are recorded in delete2

When cut-over:

  1. Count the total number of rows in the original table and record it to count1. Get the value of delete1 and calculate: res1=count1-delete1
  2. Count the total number of rows in the new table and record it to count2. Get the value of delete2 and calculate: res2=count2-delete2

When res1 is equal to res2, no data will be lost.

suyestyle avatar Nov 23 '22 03:11 suyestyle

reference only

suyestyle avatar Nov 23 '22 03:11 suyestyle

:wave: @suyestyle this might be possible to do during the atomic cut-over but it would cause a delay in the cutover. We have another approach that we use at GitHub - we continuously run test migrations against real data using replicas dedicated to testing gh-ost, which gives us confidence that we're not losing data during migrations.

ghost avatar Nov 25 '22 19:11 ghost

👋 @suyestyle this might be possible to do during the atomic cut-over but it would cause a delay in the cutover. We have another approach that we use at GitHub - we continuously run test migrations against real data using replicas dedicated to testing gh-ost, which gives us confidence that we're not losing data during migrations.

This is great.

Will this feature be added in the next stable release?

suyestyle avatar Dec 13 '22 11:12 suyestyle