starrocks icon indicating copy to clipboard operation
starrocks copied to clipboard

[Feature] Support sql digest blacklist

Open kaijianding opened this issue 2 weeks ago • 14 comments

Why I'm doing:

It's easier to blacklist a specified sql by digest rather than by regex when there are many similar complex sqls

add sql digest blacklist 389d2ef8d98994a4290b5d2e1d5838aa;
show sql digest blacklist;

ADMIN SET FRONTEND CONFIG ("enable_sql_digest"="true");
ADMIN SET FRONTEND CONFIG ("enable_sql_blacklist"="true");

select sum(gmv) from test.test_pt3;
ERROR 5202 (42000): Access denied; This sql is in blacklist, please contact your admin. Digest: 389d2ef8d98994a4290b5d2e1d5838aa

delete sql digest blacklist 389d2ef8d98994a4290b5d2e1d5838aa;

-- delete multiple digests at one time
delete sql digest blacklist d1,d2,d3

What I'm doing:

Fixes #issue

What type of PR is this:

  • [ ] BugFix
  • [x] Feature
  • [ ] Enhancement
  • [ ] Refactor
  • [ ] UT
  • [ ] Doc
  • [ ] Tool

Does this PR entail a change in behavior?

  • [ ] Yes, this PR will result in a change in behavior.
  • [x] No, this PR will not result in a change in behavior.

Checklist:

  • [x] I have added test cases for my bug fix or my new feature
  • [x] This pr needs user documentation (for new or modified features or behaviors)
    • [ ] I have added documentation for my new feature or new function
  • [ ] This is a backport pr

Bugfix cherry-pick branch check:

  • [x] I have checked the version labels which the pr will be auto-backported to the target branch
    • [x] 4.0
    • [ ] 3.5
    • [ ] 3.4
    • [ ] 3.3

[!NOTE] Introduce digest-based SQL blacklist with parsing, authorization, execution checks, SHOW/ADD/DELETE commands, and persisted state.

  • SQL Digest Blacklist (new)
    • Add SqlDigestBlackList manager with thread-safe set, verification, WAL (EditLog), and image save/load (SRMetaBlockID.DIGEST_BLACKLIST_MGR).
    • New edit log ops: OP_ADD_SQL_DIGEST_BLACK_LIST, OP_DELETE_SQL_DIGEST_BLACK_LIST with deserialization wiring.
  • Parser/AST
    • Grammar and AST for ADD SQL DIGEST BLACKLIST <digest>, DELETE SQL DIGEST BLACKLIST <digest,...>, SHOW SQL DIGEST BLACKLIST.
  • Execution & Authorization
    • StmtExecutor: handle add/delete; validate 32-hex digest; verify digest on query when blacklist/digest enabled.
    • AuthorizerStmtVisitor/RedirectStatus: enforce BLACKLIST privilege; route add/delete; show supported.
  • SHOW Output
    • ShowExecutor and ShowResultMetaFactory: list digests with metadata column Digests.
  • Global State
    • Wire into GlobalStateMgr construction, image load/save, and accessors.
  • Metrics
    • Increment COUNTER_SQL_BLOCK_HIT_COUNT on digest hit.
  • Tests
    • Add UTs covering parsing, privileges, WAL replay, image save/load, execution blocking, and operation type flags.

Written by Cursor Bugbot for commit b6bab1bc79658e45d619825103113054c69d0557. This will update automatically on new commits. Configure here.

kaijianding avatar Dec 09 '25 03:12 kaijianding

🧪 CI Insights

Here's what we observed from your CI run for b6bab1bc.

🟢 All jobs passed!

But CI Insights is watching 👀

mergify[bot] avatar Dec 09 '25 03:12 mergify[bot]

@cursor review

alvin-celerdata avatar Dec 09 '25 06:12 alvin-celerdata

The sqldigestblacklist is too long. Why not use ALTER BLACKLIST ADD BY SQLDIGEST xxxx? This way, the blacklist can be modified in multiple ways, and they're unified.

murphyatwork avatar Dec 09 '25 11:12 murphyatwork

The sqldigestblacklist is too long. Why not use ALTER BLACKLIST ADD BY SQLDIGEST xxxx? This way, the blacklist can be modified in multiple ways, and they're unified.

Currently, we have ADD BACKEND BLACKLIST DELETE BACKEND BLACKLIST SHOW BACKEND BLACKLIST

ADD COMPUTE NODE BLACKLIST DELETE COMPUTE NODE BLACKLIST SHOW COMPUTE NODE BLACKLIST

@murphyatwork How about change to ADD DIGEST BLACKLIST DELETE DIGEST BLACKLIST SHOW DIGEST BLACKLIST ?

kaijianding avatar Dec 09 '25 12:12 kaijianding

@cursor review

alvin-celerdata avatar Dec 09 '25 17:12 alvin-celerdata

@cursor review

alvin-celerdata avatar Dec 10 '25 02:12 alvin-celerdata

@cursor review

alvin-celerdata avatar Dec 10 '25 04:12 alvin-celerdata

@cursor review

alvin-celerdata avatar Dec 10 '25 06:12 alvin-celerdata

@cursor review

alvin-celerdata avatar Dec 10 '25 17:12 alvin-celerdata

@cursor review

alvin-celerdata avatar Dec 11 '25 15:12 alvin-celerdata

[Java-Extensions Incremental Coverage Report]

:white_check_mark: pass : 0 / 0 (0%)

github-actions[bot] avatar Dec 15 '25 10:12 github-actions[bot]

[FE Incremental Coverage Report]

:white_check_mark: pass : 108 / 112 (96.43%)

file detail

path covered_line new_line coverage not_covered_line_detail
:large_blue_circle: com/starrocks/qe/RedirectStatus.java 2 3 66.67% [1339]
:large_blue_circle: com/starrocks/server/GlobalStateMgr.java 3 4 75.00% [2304]
:large_blue_circle: com/starrocks/qe/StmtExecutor.java 18 19 94.74% [1959]
:large_blue_circle: com/starrocks/meta/SqlDigestBlackList.java 46 47 97.87% [70]
:large_blue_circle: com/starrocks/persist/DeleteSqlDigestBlackLists.java 3 3 100.00% []
:large_blue_circle: com/starrocks/qe/ShowResultMetaFactory.java 3 3 100.00% []
:large_blue_circle: com/starrocks/persist/EditLog.java 11 11 100.00% []
:large_blue_circle: com/starrocks/sql/analyzer/Analyzer.java 1 1 100.00% []
:large_blue_circle: com/starrocks/persist/EditLogDeserializer.java 2 2 100.00% []
:large_blue_circle: com/starrocks/sql/analyzer/AuthorizerStmtVisitor.java 10 10 100.00% []
:large_blue_circle: com/starrocks/persist/SqlDigestBlackListPersistInfo.java 3 3 100.00% []
:large_blue_circle: com/starrocks/persist/metablock/SRMetaBlockID.java 1 1 100.00% []
:large_blue_circle: com/starrocks/qe/ShowExecutor.java 5 5 100.00% []

github-actions[bot] avatar Dec 15 '25 10:12 github-actions[bot]

[BE Incremental Coverage Report]

:white_check_mark: pass : 0 / 0 (0%)

github-actions[bot] avatar Dec 15 '25 10:12 github-actions[bot]

@cursor review

alvin-celerdata avatar Dec 15 '25 17:12 alvin-celerdata

docgen

murphyatwork avatar Dec 19 '25 05:12 murphyatwork