starrocks icon indicating copy to clipboard operation
starrocks copied to clipboard

[BugFix] trigger statistics collection on UPDATE statement

Open murphyatwork opened this issue 3 weeks ago โ€ข 6 comments

Why I'm doing:

Currently, only INSERT and INSERT_OVERWRITE operations trigger statistics collection in specific cases. However, the UPDATE statement can also significantly modify data, such as UPDATE t1 SET c1 = xxxx. In such cases, the statistics should also be updated to accurately reflect these data changes.

What I'm doing:

Make the UPDATE trigger the statistics collection.

TODO:

  1. support DELETE as well
  2. change the background statistics collection healthy calculation

Fixes #issue

What type of PR is this:

  • [x] BugFix
  • [ ] Feature
  • [ ] Enhancement
  • [ ] Refactor
  • [ ] UT
  • [ ] Doc
  • [ ] Tool

Does this PR entail a change in behavior?

  • [ ] Yes, this PR will result in a change in behavior.
  • [x] No, this PR will not result in a change in behavior.

If yes, please specify the type of change:

  • [ ] Interface/UI changes: syntax, type conversion, expression evaluation, display information
  • [ ] Parameter changes: default values, similar parameters but with different default values
  • [ ] Policy changes: use new policy to replace old one, functionality automatically enabled
  • [ ] Feature removed
  • [ ] Miscellaneous: upgrade & downgrade compatibility, etc.

Checklist:

  • [ ] I have added test cases for my bug fix or my new feature
  • [ ] This pr needs user documentation (for new or modified features or behaviors)
    • [ ] I have added documentation for my new feature or new function
  • [ ] This is a backport pr

Bugfix cherry-pick branch check:

  • [x] I have checked the version labels which the pr will be auto-backported to the target branch
    • [x] 4.0
    • [x] 3.5
    • [ ] 3.4
    • [ ] 3.3

[!NOTE] Enables statistics collection for UPDATE by passing DML type into first-load triggers and updating partition/analyze logic; adds unit and SQL tests.

  • Statistics collection:
    • Pass DmlType through LoadJobStatsListener and StatisticUtils.triggerCollectionOnFirstLoad(...) to StatisticsCollectionTrigger.
    • Update StatisticsCollectionTrigger.triggerOnFirstLoad(...) to accept dmlType and handle:
      • UPDATE: collect on touched partitions regardless of version; record tablet row counts.
      • INSERT_INTO: collect only on first-load partitions (unchanged behavior).
      • Log for unsupported types (e.g., DELETE) without triggering.
    • Preserve analyze type decision; clear tablet row counts unless sampling.
  • Tests:
    • Add UT triggerOnUpdate and adapt existing tests to new signatures.
    • Add SQL tests test_update_trigger_statistics verifying FULL/SAMPLE analyze behavior after UPDATEs.

Written by Cursor Bugbot for commit dc64a100e556ddb2544ef046e826253a6d9fa04e. This will update automatically on new commits. Configure here.

murphyatwork avatar Dec 08 '25 07:12 murphyatwork

๐Ÿงช CI Insights

Here's what we observed from your CI run for 5bf682f4.

๐ŸŸข All jobs passed!

But CI Insights is watching ๐Ÿ‘€

mergify[bot] avatar Dec 08 '25 07:12 mergify[bot]

@cursor review

alvin-celerdata avatar Dec 11 '25 15:12 alvin-celerdata

@cursor review

alvin-celerdata avatar Dec 14 '25 00:12 alvin-celerdata

[Java-Extensions Incremental Coverage Report]

:white_check_mark: pass : 0 / 0 (0%)

github-actions[bot] avatar Dec 16 '25 07:12 github-actions[bot]

[FE Incremental Coverage Report]

:white_check_mark: pass : 24 / 29 (82.76%)

file detail

path covered_line new_line coverage not_covered_line_detail
:large_blue_circle: com/starrocks/statistic/StatisticUtils.java 1 3 33.33% [165, 166]
:large_blue_circle: com/starrocks/statistic/StatisticsCollectionTrigger.java 21 24 87.50% [305, 311, 315]
:large_blue_circle: com/starrocks/listener/LoadJobStatsListener.java 2 2 100.00% []

github-actions[bot] avatar Dec 16 '25 07:12 github-actions[bot]

[BE Incremental Coverage Report]

:white_check_mark: pass : 0 / 0 (0%)

github-actions[bot] avatar Dec 16 '25 07:12 github-actions[bot]

@Mergifyio backport branch-4.0

github-actions[bot] avatar Dec 16 '25 08:12 github-actions[bot]

@Mergifyio backport branch-3.5

github-actions[bot] avatar Dec 16 '25 08:12 github-actions[bot]

backport branch-4.0

โœ… Backports have been created

mergify[bot] avatar Dec 16 '25 08:12 mergify[bot]

backport branch-3.5

โœ… Backports have been created

mergify[bot] avatar Dec 16 '25 08:12 mergify[bot]