sig-transaction icon indicating copy to clipboard operation
sig-transaction copied to clipboard

Compatible issues between async commit and green GC

Open youjiali1995 opened this issue 3 years ago • 5 comments

  1. Green GC is not compatible with async commit if async apply is enabled: A lock isn't applied and the transaction may be committed.
  2. TBD

youjiali1995 avatar Dec 09 '20 08:12 youjiali1995

It's a blocker for enabling async apply by default. Maybe not a blocker for async commit GA.

sticnarf avatar Dec 09 '20 08:12 sticnarf

For ScanLock and PhysicalScanLock requests, it should update the max_ts, check the memory lock, and finally read the raftkv or the rocksdb. For checking memory locks, there are two possible implementations: wait for the memory lock being released, or return them to the client. But for the second way, the implementation should carefully keep the increasing order of the keys in the response, and do not return the found memory locks that are greater than max lock from rocksdb (to avoid missing range since tidb will continue scanning from the last key in the response).

MyonKeminta avatar Jan 19 '21 10:01 MyonKeminta

As for async apply prewrite, can we let TiKV simply return error for green gc requests if async apply prewrite is enabled?

MyonKeminta avatar Jan 21 '21 05:01 MyonKeminta

There is another corner case that green GC doesn't solve now.. https://github.com/tikv/tikv/issues/8184#issuecomment-763483003

youjiali1995 avatar Jan 21 '21 06:01 youjiali1995

Ah.. yes.. and

As for async apply prewrite, can we let TiKV simply return error for green gc requests if async apply prewrite is enabled?

I notice that this is not a good idea considering the case that TiKV's config is changed and restarted.

Recently Jay discussed with me about a new idea that maybe we can use resolved_ts to determine if there's any lock before safepoint that needs to be resolved. Since resolved_ts will also be used by follower stale read, it may be an always-running service, instead of running only if needed by CDC. I think this is a possible substitution to green gc, but sounds hard to deliver it in 5.0 GA.

MyonKeminta avatar Jan 21 '21 07:01 MyonKeminta