tidb icon indicating copy to clipboard operation
tidb copied to clipboard

`RecordHistoricalStatsToStorage` used too much memory

Open jackysp opened this issue 10 months ago • 1 comments

Bug Report

Please answer these questions before submitting your issue. Thanks!

1. Minimal reproduce step (Required)

image It seems that there are many long insert internal SQL statements in the session pool of the domain, and it appears that they have not been cleaned up properly.

2. What did you expect to see? (Required)

3. What did you see instead (Required)

4. What is your TiDB version? (Required)

v7.5.0

jackysp avatar Apr 23 '24 03:04 jackysp

Now, Historical stats have some design problems. it needs to load all stats into the memory. it is easy to lead the OOM by recording the historical stats of partition tables or wide tables. so we need to refactor it as soon as possible.

hawkingrei avatar Apr 23 '24 05:04 hawkingrei

this is issue is observed again.

image

dbsid avatar Sep 05 '24 02:09 dbsid

this is issue is observed again.

image

Now, we has disable the tidb_enable_historical_stats by default. We need to refactor this code in v9.

hawkingrei avatar Sep 05 '24 02:09 hawkingrei