starrocks icon indicating copy to clipboard operation
starrocks copied to clipboard

[BugFix] fix the behavior of fe_conf.statistic_collect_query_timeout

Open murphyatwork opened this issue 3 weeks ago โ€ข 6 comments

Why I'm doing:

The statistic_collect_query_timeout should be the timeout of the whole ANALYZE TABLE job, but actually it's treated as the timeout of a sql task.

What I'm doing:

Fixes #issue

What type of PR is this:

  • [x] BugFix
  • [ ] Feature
  • [ ] Enhancement
  • [ ] Refactor
  • [ ] UT
  • [ ] Doc
  • [ ] Tool

Does this PR entail a change in behavior?

  • [ ] Yes, this PR will result in a change in behavior.
  • [x] No, this PR will not result in a change in behavior.

If yes, please specify the type of change:

  • [ ] Interface/UI changes: syntax, type conversion, expression evaluation, display information
  • [ ] Parameter changes: default values, similar parameters but with different default values
  • [ ] Policy changes: use new policy to replace old one, functionality automatically enabled
  • [ ] Feature removed
  • [ ] Miscellaneous: upgrade & downgrade compatibility, etc.

Checklist:

  • [ ] I have added test cases for my bug fix or my new feature
  • [ ] This pr needs user documentation (for new or modified features or behaviors)
    • [ ] I have added documentation for my new feature or new function
  • [ ] This is a backport pr

Bugfix cherry-pick branch check:

  • [x] I have checked the version labels which the pr will be auto-backported to the target branch
    • [x] 4.0
    • [x] 3.5
    • [ ] 3.4
    • [ ] 3.3

[!NOTE] Enforces analyze job timeout by computing remaining time and setting per-task query/insert timeouts, updating collect APIs and adding tests.

  • Core behavior
    • Add calculateAndSetRemainingTimeout(context, analyzeStatus) in StatisticsCollectJob to compute remaining job time and set queryTimeoutS/insertTimeoutS per SQL/insert task.
    • StatisticExecutor.collectStatistics sets analyzeStatus.startTime if missing.
  • API changes
    • Update collectStatisticSync signature to collectStatisticSync(String sql, ConnectContext, AnalyzeStatus) and invoke timeout calculation before execution in:
      • FullStatisticsCollectJob, ExternalFullStatisticsCollectJob, HistogramStatisticsCollectJob, ExternalHistogramStatisticsCollectJob, SampleStatisticsCollectJob.
    • HyperStatisticsCollectJob: compute remaining timeout before each query job and before inserts; update flushInsertStatisticsData to accept AnalyzeStatus.
  • Session defaults
    • Clarify in StatisticUtils that timeouts are dynamically overridden per task, keeping large defaults.
  • Tests
    • Add testJobTimeout to verify remaining-time logic and timeout exceptions.
    • Adjust existing tests to new method signatures and start-time initialization.

Written by Cursor Bugbot for commit 8beab2136b8003c4f595f7e02e0835f85567022b. This will update automatically on new commits. Configure here.

murphyatwork avatar Dec 05 '25 06:12 murphyatwork

๐Ÿงช CI Insights

Here's what we observed from your CI run for 8beab213.

๐ŸŸข All jobs passed!

But CI Insights is watching ๐Ÿ‘€

mergify[bot] avatar Dec 05 '25 06:12 mergify[bot]

[Java-Extensions Incremental Coverage Report]

:white_check_mark: pass : 0 / 0 (0%)

github-actions[bot] avatar Dec 06 '25 04:12 github-actions[bot]

[FE Incremental Coverage Report]

:white_check_mark: pass : 24 / 29 (82.76%)

file detail

path covered_line new_line coverage not_covered_line_detail
:large_blue_circle: com/starrocks/statistic/SampleStatisticsCollectJob.java 0 2 00.00% [64, 84]
:large_blue_circle: com/starrocks/statistic/FullStatisticsCollectJob.java 0 2 00.00% [147, 190]
:large_blue_circle: com/starrocks/statistic/StatisticExecutor.java 1 2 50.00% [525]
:large_blue_circle: com/starrocks/statistic/HyperStatisticsCollectJob.java 3 3 100.00% []
:large_blue_circle: com/starrocks/statistic/ExternalFullStatisticsCollectJob.java 2 2 100.00% []
:large_blue_circle: com/starrocks/statistic/ExternalHistogramStatisticsCollectJob.java 1 1 100.00% []
:large_blue_circle: com/starrocks/statistic/StatisticsCollectJob.java 16 16 100.00% []
:large_blue_circle: com/starrocks/statistic/HistogramStatisticsCollectJob.java 1 1 100.00% []

github-actions[bot] avatar Dec 06 '25 04:12 github-actions[bot]

[BE Incremental Coverage Report]

:white_check_mark: pass : 0 / 0 (0%)

github-actions[bot] avatar Dec 06 '25 04:12 github-actions[bot]

@cursor review

alvin-celerdata avatar Dec 11 '25 15:12 alvin-celerdata

@Mergifyio backport branch-4.0

github-actions[bot] avatar Dec 16 '25 09:12 github-actions[bot]

@Mergifyio backport branch-3.5

github-actions[bot] avatar Dec 16 '25 09:12 github-actions[bot]

backport branch-4.0

โœ… Backports have been created

mergify[bot] avatar Dec 16 '25 09:12 mergify[bot]

backport branch-3.5

โœ… Backports have been created

mergify[bot] avatar Dec 16 '25 09:12 mergify[bot]