pingcap
Directly write TiKV
Feature Request
Is your feature request related to a problem? Please describe:
When we really can't meet user's requirement. 😂
Describe the feature you'd like:
Describe alternatives you've considered:
Teachability, Documentation, Adoption, Migration Strategy:
When we really can't meet user's requirement.
what?
This means that currently dm synchronises data to the downstream tidb via sql, but this is slower than writing k/v (parsed by dm itself from the upstream db) directly to the tikv.
When we really can't meet user's requirement.
what?
This means that currently dm synchronises data to the downstream tidb via sql, but this is slower than writing k/v (parsed by dm itself from the upstream db) directly to the tikv.
Is that means we need convert all sql to TiKV's commands or just handle internal schema which like checkpoints etc?
This means that currently dm synchronises data to the downstream tidb via sql, but this is slower than writing k/v (parsed by dm itself from the upstream db) directly to the tikv.
To achieve this goal, maybe we can embed TiDB server into DM as a client 🤣, or integrate DM to TiDB.
This means that currently dm synchronises data to the downstream tidb via sql, but this is slower than writing k/v (parsed by dm itself from the upstream db) directly to the tikv.
To achieve this goal, maybe we can embed TiDB server into DM as a client 🤣, or integrate DM to TiDB.
I agree with that, it's the best way for DM's software engineering!
I'm consider re-use TiDB server's "sql module"(if we could and only need send a statement tree to TiDB server. Lol) maybe is better? Cause integrate DM to TiDB would make DM as a part of TiDB server (sounds fat).
If we depend on tidb's transaction client, is the performance much better than the current logic? In most case, the bottle neck is TiKV, we may need to do some benchmark first.
We have done similar benchmark for lightning tidb backend, the performance gain is not so big.
If we depend on tidb's transaction client, is the performance much better than the current logic? In most case, the bottle neck is TiKV, we may need to do some benchmark first.
We have done similar benchmark for lightning tidb backend, the performance gain is not so big.
Great work! the reason maybe is DM works mostly as OLTP insert?