gh-ost icon indicating copy to clipboard operation
gh-ost copied to clipboard

fix retry failed bug which might result in early quit of migration

Open wfxiang08 opened this issue 7 years ago • 2 comments

Related issue: https://github.com/github/gh-ost/issues/483

Description

applyCopyRowsFunc should be retriable. But when there is error, terminateRowIteration is called, which results in a panic. So terminateRowIteration should not be called inside applyCopyRowsFunc

// Copy task:
applyCopyRowsFunc := func() error {
	if atomic.LoadInt64(&this.rowCopyCompleteFlag) == 1 {
		return nil
	}
	_, rowsAffected, _, err := this.applier.ApplyIterationInsertQuery()
	if err != nil {
		return err // terminateRowIteration shouldn't be called if you want retry
	}
	atomic.AddInt64(&this.migrationContext.TotalRowsCopied, rowsAffected)
	atomic.AddInt64(&this.migrationContext.Iteration, 1)
	return nil
}
if err := this.retryOperation(applyCopyRowsFunc); err != nil {
	return terminateRowIteration(err)
}

wfxiang08 avatar Jul 25 '18 16:07 wfxiang08

@shlomi-noach this is a simple bugfix. When there is a deadlock in ApplyIterationInsertQuery, terminateRowIteration is called and then panic happens. This make the retryOperation meaningless, and make the migrate process easy to break down.

wfxiang08 avatar Jul 25 '18 19:07 wfxiang08

@wfxiang08 oh, I'm sorry. I got a notification of you closing this issue. I must have missed it initially. I apologize for not responding to this issue.

Did you close it because you gave up, or because the fix is already done?

shlomi-noach avatar Dec 25 '18 08:12 shlomi-noach