starrocks icon indicating copy to clipboard operation
starrocks copied to clipboard

Published 20 hours ago • verified publisher icon StarRocks

[BugFix] Followers could not update statistics immediately after the statistics rebuild

Open HangyuanLiu opened this issue 2 years ago • 11 comments

What type of PR is this：

  • [x] bug
  • [ ] feature
  • [ ] enhancement
  • [ ] refactor
  • [ ] others

Which issues of this PR fixes ：

Fixes #9840

Problem Summary(Required) ：

specific reproduction conditions

  1. Run the analyze statement on the follower, the command will be forwarded to the master
  2. Collect statistical information on the master, and delete the local cache after the master collects successfully
  3. The follower runs the query statement of the related table, because the local cache is not cleared, continue to use the local cache

Checklist:

  • [x] I have added test cases for my bug fix or my new feature
  • [x] I have added user document for my new feature or new function

HangyuanLiu avatar Aug 10 '22 09:08 HangyuanLiu

run starrocks_be_build

HangyuanLiu avatar Aug 10 '22 12:08 HangyuanLiu

run starrocks_be_build

wanpengfei-git avatar Aug 11 '22 06:08 wanpengfei-git

SonarCloud Quality Gate failed.    Quality Gate failed

Bug A 0 Bugs
Vulnerability A 0 Vulnerabilities
Security Hotspot A 0 Security Hotspots
Code Smell A 0 Code Smells

0.0% 0.0% Coverage
5.0% 5.0% Duplication

sonarqubecloud[bot] avatar Aug 11 '22 11:08 sonarqubecloud[bot]

run starrocks_fe_unittest

HangyuanLiu avatar Aug 11 '22 12:08 HangyuanLiu

Could you plz explain why we could not update statistics immediately after the statistics rebuild?

Youngwb avatar Aug 12 '22 02:08 Youngwb

Could you plz explain why we could not update statistics immediately after the statistics rebuild?

specific reproduction conditions

  1. Run the analyze statement on the follower, the command will be forwarded to the master
  2. Collect statistical information on the master, and delete the local cache after the master collects successfully
  3. The follower runs the query statement of the related table, because the local cache is not cleared, continue to use the local cache

HangyuanLiu avatar Aug 12 '22 03:08 HangyuanLiu

run starrocks_fe_unittest

HangyuanLiu avatar Aug 12 '22 03:08 HangyuanLiu

run starrocks_fe_unittest

HangyuanLiu avatar Aug 12 '22 04:08 HangyuanLiu

run starrocks_fe_unittest

HangyuanLiu avatar Aug 12 '22 06:08 HangyuanLiu

[FE PR Coverage Check]

:disappointed: fail : 14 / 45 (31.11%)

file detail

path covered_line new_line coverage not_covered_line_detail
:large_blue_circle: com/starrocks/statistic/StatisticExecutor.java 0 1 00.00% [287]
:large_blue_circle: com/starrocks/statistic/StatisticsMetaManager.java 0 1 00.00% [212]
:large_blue_circle: com/starrocks/persist/EditLog.java 0 2 00.00% [803, 816]
:large_blue_circle: com/starrocks/sql/optimizer/statistics/CachedStatisticStorage.java 0 3 00.00% [116, 117, 166]
:large_blue_circle: com/starrocks/statistic/AnalyzeManager.java 0 23 00.00% [143, 144, 145, 147, 148, 149, 151, 152, 174, 175, 176, 178, 179, 180, 182, 183, 184, 300, 301, 302, 303, 304, 305]
:large_blue_circle: com/starrocks/sql/ast/ShowBasicStatsMetaStmt.java 9 10 90.00% [55]
:large_blue_circle: com/starrocks/statistic/BasicStatsMeta.java 2 2 100.00% []
:large_blue_circle: com/starrocks/sql/ast/ShowAnalyzeStatusStmt.java 1 1 100.00% []
:large_blue_circle: com/starrocks/statistic/StatisticsCollectJobFactory.java 1 1 100.00% []
:large_blue_circle: com/starrocks/load/OlapDeleteJob.java 1 1 100.00% []

wanpengfei-git avatar Aug 12 '22 06:08 wanpengfei-git

run starrocks_admit_test

wanpengfei-git avatar Aug 12 '22 07:08 wanpengfei-git