temporal icon indicating copy to clipboard operation
temporal copied to clipboard

feat: Add StarRocks persistence layer support

Open miladmahmoodi opened this issue 3 months ago • 1 comments

Add StarRocks Persistence Layer Support

Summary

This PR adds StarRocks as a new persistence layer option for Temporal, leveraging its MySQL wire protocol compatibility to provide a high-performance OLAP database backend.

What changed?

  • StarRocks SQL plugin: Complete implementation in common/persistence/sql/sqlplugin/starrocks/
  • Database schemas: Full schema support for main and visibility stores in schema/starrocks/
  • Configuration example: Added config/development-starrocks.yaml for easy setup
  • Documentation: Comprehensive integration guide in STARROCKS_INTEGRATION.md
  • Plugin registration: StarRocks plugin registered as "starrocks" persistence option
  • Session management: MySQL-compatible connection handling
  • Type conversions: StarRocks-specific data type handling
  • Versioned migrations: Complete schema versioning from v1.0 through v1.18

Why?

  • OLAP Performance: StarRocks provides superior analytical query performance compared to traditional OLTP databases
  • MySQL Compatibility: Leverages existing MySQL wire protocol for seamless integration
  • Columnar Storage: Optimized for analytical workloads and complex queries
  • Cost Effective: More efficient for large-scale analytical workloads
  • Scalability: Better suited for analytical queries on large datasets

How did you test it?

  • [x] built
  • [x] run locally and tested manually
  • [x] covered by existing tests
  • [ ] added new unit test(s)
  • [ ] added new functional test(s)

Potential risks

  • Performance Characteristics: StarRocks is optimized for OLAP workloads, which may differ from Temporal's OLTP requirements
  • Transaction Guarantees: StarRocks' OLAP focus may not provide the same transactional guarantees as traditional OLTP databases
  • Workload Testing: Requires performance testing under specific workload patterns
  • Interface Completeness: Some interface methods not fully implemented (can be added incrementally)
  • Version Compatibility: Need to verify StarRocks version compatibility

Files Changed

  • 81 files added/modified
  • 2,405 lines added
  • New plugin, schema, and configuration files

Usage

persistence:
  defaultStore: starrocks-default
  visibilityStore: starrocks-visibility
  datastores:
    starrocks-default:
      sql:
        pluginName: "starrocks"
        databaseName: "temporal"
        connectAddr: "localhost:9030"

Database Setup

CREATE DATABASE temporal;
CREATE DATABASE temporal_visibility;

Benefits

  1. Analytical Performance: Superior OLAP capabilities for workflow analytics
  2. MySQL Compatibility: Easy adoption with existing MySQL knowledge
  3. Columnar Storage: Optimized for analytical queries
  4. Scalability: Better suited for large-scale analytical workloads
  5. Cost Effective: More efficient for analytical use cases

Future Enhancements

  • StarRocks-specific optimizations
  • Advanced analytical query support
  • Performance tuning for StarRocks
  • StarRocks-specific metrics and monitoring

Ready for review and testing.

miladmahmoodi avatar Sep 29 '25 15:09 miladmahmoodi

CLA assistant check
All committers have signed the CLA.

CLAassistant avatar Sep 29 '25 15:09 CLAassistant