starrocks icon indicating copy to clipboard operation
starrocks copied to clipboard

[Enhancement] opensource staros - java code

Open kevincai opened this issue 2 months ago โ€ข 16 comments

  • open source staros java source code

Why I'm doing:

What I'm doing:

#63826

What type of PR is this:

  • [ ] BugFix
  • [ ] Feature
  • [x] Enhancement
  • [ ] Refactor
  • [ ] UT
  • [ ] Doc
  • [ ] Tool

Does this PR entail a change in behavior?

  • [ ] Yes, this PR will result in a change in behavior.
  • [x] No, this PR will not result in a change in behavior.

If yes, please specify the type of change:

  • [ ] Interface/UI changes: syntax, type conversion, expression evaluation, display information
  • [ ] Parameter changes: default values, similar parameters but with different default values
  • [ ] Policy changes: use new policy to replace old one, functionality automatically enabled
  • [ ] Feature removed
  • [ ] Miscellaneous: upgrade & downgrade compatibility, etc.

Checklist:

  • [ ] I have added test cases for my bug fix or my new feature
  • [ ] This pr needs user documentation (for new or modified features or behaviors)
    • [ ] I have added documentation for my new feature or new function
  • [ ] This is a backport pr

Bugfix cherry-pick branch check:

  • [x] I have checked the version labels which the pr will be auto-backported to the target branch
    • [ ] 4.0
    • [ ] 3.5
    • [ ] 3.4
    • [ ] 3.3

[!NOTE] Introduce the StarOS Java implementation with shard/worker management, gRPC agent, placement/scheduling, file stores, utilities, and comprehensive tests.

  • Starlet (gRPC) Agent:
    • StarletAgent with heartbeat, add/remove shard RPCs, retries, and channel/stub lifecycle.
    • Mock agents for unit tests.
  • Worker Management:
    • Worker, WorkerGroup, WorkerManager with create/update/delete, heartbeats, dump/load image, and dynamic thread-pool sizing.
    • Resource provisioning via DefaultResourceManager (external or builtin test server) and factory.
  • Shard Management:
    • Shard, ShardGroup, MetaGroup (PACK/SPREAD/EXCLUDE), anonymous group handling, and meta-group operations.
    • Health/balance checker and ShardSchedulerV2 (add/remove to worker/group, batch, balancing, temp replicas).
    • ShardPolicyFilter with EXCLUDE policy.
  • File Store/Path:
    • S3/HDFS/Azure Blob/ADLS2/GS stores, replace/update flows, path and checksum utilities.
  • Utilities/Infra:
    • Config, logging, ID generator, abstract server, metrics, log dumping tool (ReadImage).
  • HTTP/Service:
    • HTTP dispatcher/service endpoints; StarManager orchestration.
  • Tests:
    • Extensive unit tests covering managers, scheduler, shard/replica states, file stores, HTTP, journal replayer, and provisioner.

Written by Cursor Bugbot for commit dfe707903f9e973ed0aec61dcf306e1cfd5b59fb. This will update automatically on new commits. Configure here.

kevincai avatar Oct 09 '25 03:10 kevincai

@cursor review

alvin-celerdata avatar Oct 09 '25 05:10 alvin-celerdata

@cursor review

alvin-celerdata avatar Nov 06 '25 21:11 alvin-celerdata

๐Ÿงช CI Insights

Here's what we observed from your CI run for dfe70790.

๐ŸŸข All jobs passed!

But CI Insights is watching ๐Ÿ‘€

mergify[bot] avatar Dec 04 '25 14:12 mergify[bot]

@cursor review

alvin-celerdata avatar Dec 04 '25 17:12 alvin-celerdata

@cursor review

alvin-celerdata avatar Dec 05 '25 17:12 alvin-celerdata

@cursor review

alvin-celerdata avatar Dec 08 '25 03:12 alvin-celerdata

@mergifyio rebase

kevincai avatar Dec 09 '25 03:12 kevincai

rebase

โœ… Branch has been successfully rebased

mergify[bot] avatar Dec 09 '25 03:12 mergify[bot]

@cursor review

alvin-celerdata avatar Dec 09 '25 06:12 alvin-celerdata

@mergifyio rebase

kevincai avatar Dec 09 '25 06:12 kevincai

rebase

โœ… Branch has been successfully rebased

mergify[bot] avatar Dec 09 '25 06:12 mergify[bot]

Quality Gate Failed Quality Gate failed

Failed conditions
10 Security Hotspots
E Reliability Rating on New Code (required โ‰ฅ A)

See analysis details on SonarQube Cloud

Catch issues before they fail your Quality Gate with our IDE extension SonarQube for IDE

sonarqubecloud[bot] avatar Dec 09 '25 06:12 sonarqubecloud[bot]

[Java-Extensions Incremental Coverage Report]

:white_check_mark: pass : 0 / 0 (0%)

github-actions[bot] avatar Dec 09 '25 09:12 github-actions[bot]

[FE Incremental Coverage Report]

:white_check_mark: pass : 0 / 0 (0%)

github-actions[bot] avatar Dec 09 '25 09:12 github-actions[bot]

[BE Incremental Coverage Report]

:white_check_mark: pass : 0 / 0 (0%)

github-actions[bot] avatar Dec 09 '25 09:12 github-actions[bot]

@cursor review

alvin-celerdata avatar Dec 09 '25 17:12 alvin-celerdata