starrocks icon indicating copy to clipboard operation
starrocks copied to clipboard

[Refactor] Transform User/Role related edit logs to WAL format

Open gengjun-git opened this issue 2 weeks ago โ€ข 9 comments

Why I'm doing:

What I'm doing:

Refer https://github.com/StarRocks/starrocks/issues/63357

What type of PR is this:

  • [ ] BugFix
  • [ ] Feature
  • [ ] Enhancement
  • [x] Refactor
  • [ ] UT
  • [ ] Doc
  • [ ] Tool

Does this PR entail a change in behavior?

  • [ ] Yes, this PR will result in a change in behavior.
  • [x] No, this PR will not result in a change in behavior.

If yes, please specify the type of change:

  • [ ] Interface/UI changes: syntax, type conversion, expression evaluation, display information
  • [ ] Parameter changes: default values, similar parameters but with different default values
  • [ ] Policy changes: use new policy to replace old one, functionality automatically enabled
  • [ ] Feature removed
  • [ ] Miscellaneous: upgrade & downgrade compatibility, etc.

Checklist:

  • [ ] I have added test cases for my bug fix or my new feature
  • [ ] This pr needs user documentation (for new or modified features or behaviors)
    • [ ] I have added documentation for my new feature or new function
  • [ ] This is a backport pr

Bugfix cherry-pick branch check:

  • [x] I have checked the version labels which the pr will be auto-backported to the target branch
    • [ ] 4.0
    • [ ] 3.5
    • [ ] 3.4
    • [ ] 3.3

[!NOTE] Switches user/role authorization edit logs to WAL with in-memory appliers, adopts copy-on-write via deep cloning, refines user property updates, and adds comprehensive tests.

  • Authorization & Authentication (RBAC):
    • Adopt copy-on-write (COW) updates using .clone() for UserPrivilegeCollectionV2 and RolePrivilegeCollectionV2 across grant/revoke, role grant/revoke, default-role updates, and role create/alter/drop.
    • Add setUserPrivilegeCollection and ensure cache invalidation is done post-WAL.
    • Make built-in roles immutable during WAL replay via replacePrivilegeEntryForImmutableBuiltInRole.
  • Persistence (WAL):
    • Convert edit log APIs to WAL-style with applier callbacks: logCreateUser, logAlterUser, logUpdateUserPropertyV2, logDropUser, logUpdateUserPrivilege, logUpdateRolePrivilege, logDropRole, and security integration/group provider logs.
    • EditLog.loadJournal updated to replay new info objects (CreateUserInfo, AlterUserInfo, UserPrivilegeCollectionInfo, RolePrivilegeCollectionInfo).
  • User management:
    • AuthenticationMgr: wrap create/alter/drop user and property updates in WAL; update in-memory state only in WAL appliers; minor logging cleanups.
  • UserProperty:
    • Split validation/apply into checkUpdate (returns UpdateInfo) and update(UpdateInfo), plus keep update(List<...>); adjust replay to be exception-safe.
  • Cloning utilities:
    • Add deep-clone helpers: PrivilegeCollectionV2.cloneTypeToPrivilegeEntryList(), override clone() in RolePrivilegeCollectionV2, UserPrivilegeCollectionV2, and add clone() to ViewPEntryObject/MaterializedViewPEntryObject.
  • Tests:
    • Add extensive edit-log/WAL and cloning tests for AuthenticationMgr and AuthorizationMgr; update existing tests to new APIs and behaviors.

Written by Cursor Bugbot for commit e0848c841046aa48d5465696bf6702f296f33a8e. This will update automatically on new commits. Configure here.

gengjun-git avatar Dec 09 '25 08:12 gengjun-git

๐Ÿงช CI Insights

Here's what we observed from your CI run for 64f07da5.

๐ŸŸข All jobs passed!

But CI Insights is watching ๐Ÿ‘€

mergify[bot] avatar Dec 09 '25 08:12 mergify[bot]

@cursor review

alvin-celerdata avatar Dec 09 '25 17:12 alvin-celerdata

@cursor review

gengjun-git avatar Dec 10 '25 08:12 gengjun-git

@cursor review

alvin-celerdata avatar Dec 10 '25 17:12 alvin-celerdata

@cursor review

alvin-celerdata avatar Dec 11 '25 02:12 alvin-celerdata

@cursor review

alvin-celerdata avatar Dec 12 '25 17:12 alvin-celerdata

[Java-Extensions Incremental Coverage Report]

:white_check_mark: pass : 0 / 0 (0%)

github-actions[bot] avatar Dec 15 '25 07:12 github-actions[bot]

[FE Incremental Coverage Report]

:x: fail : 25 / 32 (78.12%)

file detail

path covered_line new_line coverage not_covered_line_detail
:large_blue_circle: com/starrocks/persist/EditLog.java 0 2 00.00% [2055, 2059]
:large_blue_circle: com/starrocks/authentication/UserProperty.java 0 1 00.00% [73]
:large_blue_circle: com/starrocks/authentication/AuthenticationMgr.java 25 29 86.21% [271, 323, 342, 343]

github-actions[bot] avatar Dec 15 '25 07:12 github-actions[bot]

[BE Incremental Coverage Report]

:white_check_mark: pass : 0 / 0 (0%)

github-actions[bot] avatar Dec 15 '25 08:12 github-actions[bot]

@mergify rebase

gengjun-git avatar Dec 15 '25 10:12 gengjun-git

rebase

โŒ Base branch update has failed

Git reported the following error:

Rebasing (1/13)
Rebasing (2/13)
Rebasing (3/13)
Rebasing (4/13)
Rebasing (5/13)
Rebasing (6/13)
Rebasing (7/13)
Rebasing (8/13)
Auto-merging fe/fe-core/src/test/java/com/starrocks/authentication/AuthenticationMgrEditLogTest.java
CONFLICT (content): Merge conflict in fe/fe-core/src/test/java/com/starrocks/authentication/AuthenticationMgrEditLogTest.java
error: could not apply e3a8cee80b... add ut
hint: Resolve all conflicts manually, mark them as resolved with
hint: "git add/rm <conflicted_files>", then run "git rebase --continue".
hint: You can instead skip this commit: run "git rebase --skip".
hint: To abort and get back to the state before "git rebase", run "git rebase --abort".
Could not apply e3a8cee80b... add ut

mergify[bot] avatar Dec 15 '25 10:12 mergify[bot]