tidb icon indicating copy to clipboard operation
tidb copied to clipboard

planner: Memory tracking for plan cache

Open fzzf678 opened this issue 3 years ago • 0 comments

Enhancement

  • Currently, there is no interface to get the memory usage of plan cache. We can only estimate the memory each plan used and set a limit number to control the memory.It is imprecise and may cause OOM of TiDB.So we would like to add memory tracker for plan cache to count and manage the memory usage.

Items

PhysicalPlan memory trace

  • [x] Expression(Column/Constant/CorrelatedColumn/ScalarFunction) memory trace https://github.com/pingcap/tidb/pull/37624
  • [x] PhysicalPlan(sort operator) memory trace https://github.com/pingcap/tidb/pull/37655
  • [x] PhysicalPlan(scan operator) memory trace https://github.com/pingcap/tidb/pull/37668
  • [x] PhysicalPlan(agg/limit/select/projection operator) memory trace https://github.com/pingcap/tidb/pull/37731
  • [x] PhysicalPlan(reader operator) memory trace https://github.com/pingcap/tidb/pull/37736
  • [x] PhysicalPlan(join operator) memory trace https://github.com/pingcap/tidb/pull/37860
  • [ ] PhysicalPlan(pointGet/apply/topN operator) memory trace https://github.com/pingcap/tidb/pull/37841
  • [ ] PhysicalPlan(mock/CTE )https://github.com/pingcap/tidb/pull/38030
  • [ ] Implement the rest PhysicalPlan operator(....)

Count other memory usage in plan cache

  • [ ] LRUPlanCache、planCacheKey、PlanCacheValue memory trace https://github.com/pingcap/tidb/pull/38069
  • [ ] CachedPrepareStmt、ParamMarker memory trace

Test and attach

  • [ ] Test accuracy of plan cache memory trace and correct
  • [ ] Access memTracker

fzzf678 avatar Sep 06 '22 02:09 fzzf678