[Sandbox] StarRocks

[alberttwong]

trafficstars

Application contact emails

[email protected], [email protected], [email protected]

Project Summary

StarRocks, a Linux Foundation project, is a next-generation sub-second MPP OLAP database for full analytics scenarios, including multi-dimensional analytics, real-time analytics, and ad-hoc queries.

Project Description

The StarRocks project is an open source, distributed, MPP (Massively Parallel Processing) OLAP database that is designed for high performance and scalability.

StarRocks is needed because there is a growing demand for high-performance OLAP databases that can handle large amounts of data. Traditional OLAP databases, such as Snowflake, Microsoft SQL Server, GCP BigQuery and AWS Redshift, are not designed to handle the massive amounts of data that are generated by today's applications in real time. StarRocks is designed to address this challenge by providing high performance and scalability for large-scale real time data analytics.

Here are some of the key features of StarRocks:

• High performance: StarRocks can process queries on large datasets in milliseconds.

• Scalability: StarRocks can be scaled out to handle even the most demanding workloads.

• Flexibility: StarRocks supports a wide range of data types and can be used for a variety of data analytics tasks.

• Open source: StarRocks is open source, which means that it is free to use and modify.

The StarRocks project is still under development, but it has already been adopted by a number of organizations, including AirBnb, Alibaba, Tencent, and JD.com. It is a promising new OLAP database that has the potential to revolutionize the way we analyze data.

Org repo URL (provide if all repos under the org are in scope of the application)

https://github.com/StarRocks/

Project repo URL in scope of application

https://github.com/StarRocks/starrocks

Additional repos in scope of the application

No response

Website URL

https://www.starrocks.io/

Roadmap

https://github.com/StarRocks/starrocks/issues/16445

Roadmap context

No response

Contributing Guide

https://github.com/StarRocks/starrocks/blob/main/CONTRIBUTING.md

Code of Conduct (CoC)

https://github.com/StarRocks/starrocks/blob/main/CODE_OF_CONDUCT.md

Adopters

No response

Contributing or Sponsoring Org

https://celerdata.com/

Maintainers file

https://github.com/StarRocks/starrocks/blob/main/community/membership.md

IP Policy

• [X] If the project is accepted, I agree the project will follow the CNCF IP Policy

Trademark and accounts

• [X] If the project is accepted, I agree to donate all project trademarks and accounts to the CNCF

Why CNCF?

StarRocks can contribute to the CNCF in a number of ways, including:

• Providing a high-performance, scalable, and flexible OLAP database for cloud-native applications. StarRocks is designed to meet the needs of modern data analytics workloads, which are often characterized by large datasets, complex queries, and real-time requirements.

• Helping to advance the state of cloud-native database technology. StarRocks is an open source project, which means that it is developed and maintained by a community of contributors. This allows for rapid innovation and collaboration, which can help to advance the state of cloud-native database technology.

• Growing the cloud-native database ecosystem. The CNCF is home to a number of other cloud-native database projects, such as TiDB, Vitess, and CockroachDB. By joining the CNCF, StarRocks can help to grow the cloud-native database ecosystem and make it more accessible to developers.

Being in the CNCF would provide the StarRocks project with a number of benefits, including:

• Access to a large and active community of developers. The CNCF has a large and active community of developers who are interested in cloud-native technologies. This can help to attract new contributors to the StarRocks project and ensure that it is well-maintained.

• Access to resources and support from the CNCF. The CNCF provides a number of resources and support to its member projects, such as funding, training, and marketing. This can help the StarRocks project to grow and succeed.

• Increased visibility and adoption. Being a member of the CNCF can help to increase the visibility and adoption of StarRocks. This can lead to more users and contributors, which can help to further develop and improve the project.

Overall, StarRocks can contribute to the CNCF and benefit from being a member of the CNCF in a number of ways. This can help to make StarRocks a more successful project and make it more accessible to developers.

Benefit to the Landscape

This would provide another option for real time analytics OLAP database in the marketplace.

Cloud Native 'Fit'

We see ourselves as a cloud native database to deployed in k8s and other cloud technologies.

Cloud Native 'Integration'

k8s, containers and all the other projects in the k8s community.

Cloud Native Overlap

No response

Similar projects

https://landscape.cncf.io/card-mode?category=database&grouping=category

Landscape

Yes at https://landscape.cncf.io/card-mode?category=database&grouping=category&selected=star-rocks

Business Product or Service to Project separation

As a Linux Foundation project, we have already done the work to separate the community products from the sponsoring company.

Project presentations

No response

Project champions

No response

Additional information

No response

[jberkus]

Given the name, can I assume that the base storage is RocksDB?

[alberttwong]

Given the name, can I assume that the base storage is RocksDB?

No. It's our own storage. No code from RocksDB

[jberkus]

When I look at StarRocks, I see an analytics database that's definitely useful in a cloud-native context, but which isn't itself cloud-native, any more than, say, CitusDB or Snowflake would be. Can you explain what makes StarRocks cloud native? There's sections for that in the application, but you haven't filled them out.

[alberttwong]

Before I get to the explain part, how do you define cloud native?

[TheFoxAtWork]

The CNCF TOC maintains the definition of cloud native for the CNCF in our Repository: https://github.com/cncf/toc/blob/main/DEFINITION.md

[alberttwong]

When I look at StarRocks, I see an analytics database that's definitely useful in a cloud-native context, but which isn't itself cloud-native, any more than, say, CitusDB or Snowflake would be. Can you explain what makes StarRocks cloud native? There's sections for that in the application, but you haven't filled them out.

Based on cloud native definition, we have a k8s operator that you can deploy our solution anywhere k8s and containers are supported.

[alberttwong]

@jberkus @TheFoxAtWork I was looking at the CNCF review schedule at https://github.com/orgs/cncf/projects/27 and https://github.com/orgs/cncf/projects/14. Do you have an estimate on when will we be reviewed and is there any way for us to expedite the process?

[amye]

I was looking at the CNCF review schedule at https://github.com/orgs/cncf/projects/27 and https://github.com/orgs/cncf/projects/14. Do you have an estimate on when will we be reviewed and is there any way for us to expedite the process? Next review is January 23rd, we'll move projects that are coming up for review then. As we pause on all projects moving levels or coming in six weeks before Kubecon, April 9th is the following review meeting. That's truly as fast as we can do these!

[nikhita]

xref - https://github.com/cncf/toc/issues/889 (since this is already a LF project)

[rochaporto]

Was looking for a reference, did we get a presentation of the project to a TAG? TAG-Storage? @xing-yang

[xing-yang]

@rochaporto We have not seen a presentation from StarRocks at TAG-Storage yet. @alberttwong let us know when you want to present. TAG-Storage meets on the 2nd and 4th Wednesday of every month at 8AM PT. Thanks.

[alberttwong]

@xing-yang no one told us of this requirement. What is TAG-Storage and what type of presentation do you need to have? Can you contact me at [email protected]

[amye]

Presenting to a TAG is optional for sandbox projects, it provides a better perspective for the TOC. Given as this is scheduled for review next week, best to have the TAG Storage folks ask questions in here instead of trying to schedule a presention. @xing-yang @chira001 for awareness

[xing-yang]

@alberttwong I have sent out an email to you with more details, but I just saw the message from @amye.

[alberttwong]

@amye do you know what we need to prepare for our April 9th meeting? I haven't seen an invite (detail to join and time) or what we need to prepare.

[amye]

The TOC meets in a closed meeting to review and discuss, the recording will be published on the CNCF's YouTube channel. Votes open after that meeting, they close a week later, on April 16th. Nothing the projects need to do.

[kevin-wangzefeng]

Curious about the trademark status of StarRocks: According to the LICENSE file, it is StarRocks, Inc. holding the copyright of this project. What is the status of StarRocks, Inc.? Though this is already a LF project, I can't find it in https://www.linuxfoundation.org/legal/trademarks

Ref: https://github.com/StarRocks/starrocks/blob/35f61f77c21a37dc8347c5923501918bb0170667/LICENSE.txt#L1C1-L2C1

[alberttwong]

What is the status of StarRocks, Inc.

I'll have to ask [email protected] our contact at LF.

[alberttwong]

@kevin-wangzefeng I have no response from kristi (ktan@). Do you know anyone else at LF that we could contact?

[jeefy]

Is this just an artifact that needs to be removed from the license since this project is already under the LF?

Also Kristi no longer works for the LF, so your emails probably bounced. :(

[alberttwong]

@jeefy do you know who is Kristi's replacement? I also emailed [email protected] and [email protected] and I've gotten no reply.

[alberttwong]

@kevin-wangzefeng @jeefy I just got a reply.

Hi, Albert, you are correct. Just tell CNCF that the name is held by LF Projects, LLC and ask them to connect with me if they have any questions. Thank you,

• Scott -- Scott A. Nicholas Vice President of Project Formation, The Linux Foundation

[alberttwong]

@amye has the video of the meeting been published?

[amye]

https://youtu.be/h63Sg_qDQT8?si=hcknyYV7vSFyom8l - it's in the TOC playlist on the CNCF YouTube channel

[alberttwong]

I saw 2 concerns in the meeting.

1. Licensing. I believe this is resolved through my cut and paste of my communication with Scott A. Nicholas @ LF
2. TAG review. I have that scheduled but can you modify the application process so that future projects know that they have go through a TAG review?

[alberttwong]

@mauilion Can you confirm receipt and resolution of the legal concern via my communication with Scott A. Nicholas @ LF

[jberkus]

If StarRocks already belongs to the Linux Foundation, then what's the motive to move it into the CNCF? It's not really cloud native software.

[creatstar]

@jberkus Thank you for asking. StarRocks is a cloud-native analytical database.

1. StarRocks' architecture of separating storage and computation supports storing data in various types of cloud service object storage.
2. StarRocks supports direct queries on the most popular open data lakes today, and it is compatible with cloud catalog services like AWS Glue.
3. StarRocks supports deployment and management using the K8S connector.

StarRocks' motivation for joining CNCF is to collaborate with other CNCF projects and better leverage the various features of the cloud, providing a superior data analytics experience for enterprise customers

[jberkus]

@creatstar

I'm not sure you that you understand what we mean when we say "cloud native". Particularly, we're looking for integration into a stack of open source container-based microservice platforms -- not compatibility with AWS. As far as I can tell, StarRocks doesn't support deployment on Kubernetes and it doesn't integrate with other CNCF projects.

StarRocks is a great database, and I'm really glad that it's under LF stewardship. But it's a very poor fit for the CNCF, and joining would just make both the CNCF folks and the StarRocks folks frustrated. (this is my personal opinion as a database geek; I am not a member of the TOC and do not speak for them)

[alberttwong]

@jberkus Maybe I didn't make it clear. When we say that we have an operator, it is a kubernetes operator. That means that it supports deployment on kubernetes and relies kubernetes network and the horizontal scaling of containers is managed by kubernetes and request storage from kubernetes PV/PVCs.