sandbox

sandbox copied to clipboard

Application contact emails

[email protected], [email protected], [email protected]

Project Summary

Connect is a simple, cross-language framework for Protobuf RPC.

Project Description

Connect is a cross-language framework for building strongly-typed APIs. It's most commonly used with Protocol Buffers, and implementations are currently available in Go, TypeScript, Swift, and Kotlin.

Connect expands the appeal of strongly-typed, schema-first API development without further fragmenting the RPC ecosystem.

1. Connect implementations are small, which makes them stable, reliable, and debuggable.
2. Along with Connect's own protocol, implementations fully support the gRPC and gRPC-Web protocols. Servers support ingress from all three protocols, and clients can switch protocols with a single configuration flag. Connect's own protocol works over HTTP/1.1 and HTTP/2, supports request-response and streaming RPCs, and is easy to use from web browsers and curl.
3. Connect builds on familiar primitives, like net/http in Go, URLSession in Swift, and Fastify, Next.js, and Express in Node. This makes Connect approachable and easy to integrate into existing applications, and it lets users take advantage of these frameworks' mature ecosystems.

Connect is currently in production at companies of all sizes. Adopters include Buf, PlanetScale, RedPanda, Grafana Labs, GitPod, Chick-fil-A, BlueSky, Dropbox, and a multi-national bank. In most of these deployments, Connect runs alongside and interoperates with gRPC systems.

Org repo URL (provide if all repos under the org are in scope of the application)

https://github.com/connectrpc

Project repo URL in scope of application

N/A

Additional repos in scope of the application

No response

Website URL

https://connectrpc.com

Roadmap

https://github.com/orgs/connectrpc/discussions/16

Roadmap context

Most of Connect's medium-term roadmap is continued refinement of the Go, TypeScript, Swift, and Kotlin implementations. The specific work planned in each of those repositories is outlined in their issues. The linked GitHub discussion only covers planned work which affects the Connect project as a whole.

(Each repository also has its own CONTRIBUTING.md - the guide linked below is for the documentation and governance repository.)

Contributing Guide

https://github.com/connectrpc/connectrpc.com/blob/main/.github/CONTRIBUTING.md

Code of Conduct (CoC)

https://github.com/connectrpc/connectrpc.com/blob/main/.github/CODE_OF_CONDUCT.md

Adopters

No response

Contributing or Sponsoring Org

https://buf.build

Maintainers file

https://github.com/connectrpc/connectrpc.com/blob/main/MAINTAINERS.md

IP Policy

• [X] If the project is accepted, I agree the project will follow the CNCF IP Policy

Trademark and accounts

• [X] If the project is accepted, I agree to donate all project trademarks and accounts to the CNCF

Why CNCF?

To grow sustainably as an open source project, Connect needs input and investment from its community. The CNCF provides a vendor-neutral home that gives everyone - from individuals to large companies - confidence that Connect is open to community feedback, governed well, and responsible with its intellectual property. In short, we hope that CNCF membership allows Connect to outgrow Buf.

Benefit to the Landscape

Connect makes efficient, schema-first RPC APIs appealing to a broader range of developers, many of whom would otherwise choose free-form REST+JSON. As noted above, Connect implementations are:

• Simple, and therefore stable, reliable, and debuggable;
• Compatible with HTTP/1.1 and HTTP/2;
• Ergonomic to call from web browsers and curl; and
• Easily integrated with popular frameworks.

Notably, Connect achieves these goals without further fragmentation - Connect servers and clients are also fully compatible with the gRPC ecosystem. Ultimately, Connect makes cloud native architectures accessible to more developers, more projects, and more companies.

Cloud Native 'Fit'

Connect naturally fits into the Remote Procedure Call portion of the CNCF landscape: it lets developers build efficient, schema-first APIs, the bedrock of every cloud native architecture.

Cloud Native 'Integration'

Connect doesn't depend on any CNCF projects, but it complements several:

• gRPC: Connect servers and clients interoperate seamlessly with the gRPC ecosystem.
• Envoy: recent Envoy releases include filters that collect metrics for Connect RPCs and allow clients using Connect's protocol to transparently call gRPC servers.
• OpenTelemetry: like all schema-first RPC frameworks, Connect's design lends itself to rich out-of-the-box observability. The Connect project includes OpenTelemetry interceptors that follow the recommended semantic conventions.

Cloud Native Overlap

To at least some extent, Connect overlaps with all the CNCF's RPC projects. It's closest to gRPC - both are cross-language frameworks commonly paired with Protocol Buffers - but the two projects complement each other well.

Similar projects

Within the CNCF, just gRPC. Elsewhere:

• Thrift and FBThrift
• Twirp
• Dubbo
• Cap'n Proto, dRPC, fRPC, SRPC, TARS and many more (including the rest of the RPC portion of the CNCF landscape).

Landscape

Not yet.

Business Product or Service to Project separation

Connect is sponsored by Buf, whose other projects include a Protocol Buffer compiler and a schema registry. However, Connect is a separate project, does not share a name or logo with any other Buf products, and is developed in a separate GitHub organization with an open governance structure. Several of Connect's most enthusiastic contributors are now maintainers, with voices (and votes!) equal to those of Buf-employed maintainers.

Project presentations

No response

Project champions

No response

Additional information

No response

10/12: TAG network team reviewed this, and recommended the team to beef up cloud native relationships to their site. Team recommended CNCF to accept the project as CNCF sandbox project.

Please present to TAG App Delivery on this project.

@linsun can you provide any clarification wrt how this project fits into the scope of TAG Network.

@akshayjshah will you update the integrations to reflect the current state of integration with cloud native projects?

@TheFoxAtWork Sure, happy to present to TAG App Delivery! I'm now on the schedule to present on Jan 17.

I'm a little surprised by this bucketing, though. Do other RPC projects like gRPC, Thrift, and Dubbo roll up to App Delivery? From the tagline ("building, packaging, deploying, managing, and operating" cloud-native applications) and the other projects in their portfolio (Helm, Buildpacks, ArgoCD, Harbor, etc.), I wouldn't have immediately picked App Delivery as the right group. I certainly can't speak for @linsun, but TAG Network felt like the more natural home for an RPC project - the description for TAG Network even mentions that the Remote Procedure Call section of the landscape is in their scope.

@akshayjshah will you update the integrations to reflect the current state of integration with cloud native projects?

It's up to date :)

@akshayjshah I completely understand - we're currently working to ensure projects have the correct visibility across TAGs as well as ensure they are aligned or homed under a primary TAG. I'm sure both TAGs would appreciate project specific insights as they explore their current charter, description, and domain scope, particularly as we're looking to refresh these in 2024.

Apologies for my lack of clarity in the previous comment regarding integrations, both gRPC and SPIFFE/SPIRE are mentioned but the project does not appear to integrate with either, rather it may be used in conjunction with SPIFFE/SPIRE (as i understand it) or in lieu of gRPC (as a more simplistic and focused implementation). This post from Buf mentions some of the difficulty around gRPC and its challenges, calls out etcd as having been subject to broken compatibility as a result of those challenges. Is Connect looking to collaborate and connect with etcd in the future to alleviate those challenges as a gRPC alternative? what other cloud native projects is the project planning on integrating with?

No problem - I'm happy to present to TAG App Delivery if they're potentially involved in RPC going forward. Just wanted to make sure there weren't any wires crossed here :)

both gRPC and SPIFFE/SPIRE are mentioned but the project does not appear to integrate with either

I must have misunderstood this question on the application. The form asks, "What CNCF projects does this project complement or depend on, and how?" I thought it was asking about Connect's dependencies, and which existing projects Connect complements. Connect implementations are too low-level to have dependencies on other CNCF projects. We do think that Connect complements gRPC, but of course neither project depends on the other's code. (This is the same relationship that Envoy and gRPC have: they speak the same protocol, but Envoy recommends using its own implementation rather than gRPC's C++ libraries. linkerd also built their own gRPC implementation, but theirs is factored out into a separate project.)

I did remove SPIFFE/SPIRE.

Is Connect looking to collaborate and connect with etcd in the future to alleviate those challenges as a gRPC alternative?

We'd love to work with any CNCF project. In the abstract, etcd is a good fit for Connect: they serve a gRPC API, but they also serve an HTTP+JSON API for clients that can't use HTTP/2 and trailers. Connect solves those problems with a minimum of fuss.

In practice, though, we're pragmatists. Working software is invaluable, and etcd has become mission-critical to many projects and organizations. If the etcd maintainers ever consider a v4, bundling in a migration to Connect might make sense. In the short term, it seems unnecessarily risky to swap out such a foundational part of the project. (I did have this conversation with a few of the etcd maintainers at KubeCon NA this year.)

Some earlier-stage CNCF projects are choosing Connect, though - I just saw that OpenFeature's flagd uses Connect. We'd love to work with more similar projects in the future.

Please present to TAG App Delivery on this project.

@linsun can you provide any clarification wrt how this project fits into the scope of TAG Network.

Hi @TheFoxAtWork, gRPC currently is hosted in TAG-network, which is similar to Connect, e.g. fits into the Remote Procedure Call function area outlined in tag network home page.

Just a quick update from today's TAG network meeting:

Recommendation is for the Connect team (Akshay) to present to TAG app delivery in Jan 2024 but we still believe this should fit into tag network given gRPC is in TAG network.

Agreed.

Thanks all. It makes sense to ask Connect to present their framework to TAG App Delivery, as a number of API and communication protocol frameworks have recently been supported by our group, including Microcks (#37), Apicurio, and a WG for API patterns and standards (see https://github.com/cncf/tag-app-delivery/issues/448).

@akshayjshah as you suggested it would be great to talk at our January 17 meeting. I opened https://github.com/cncf/tag-app-delivery/issues/520 in our repo to track the discussion.

Hi @akshayjshah - I'd like to confirm with you that we're on for next week at TAG App Delivery's general meeting. We meet on Wednesday 1/17 at 1600 UTC, Zoom link is at https://tag-app-delivery.cncf.io/. Looking forward to the discussion!

Can you reply here or in https://github.com/cncf/tag-app-delivery/issues/520 to confirm and with any questions? Thank you!

Can you reply here or in https://github.com/cncf/tag-app-delivery/issues/520 to confirm and with any questions? Thank you!

Confirmed! I'm planning a short presentation.

Thank you @akshayjshah for presenting Connect to TAG App Delivery. The TAG agreed that Connect fits in CNCF alongside the likes of GRPC in providing an API/RPC framework for cloud-native application development. See https://github.com/cncf/tag-app-delivery/issues/520#issuecomment-1904510720 for our notes and a link to the recording.

It also does make sense that TAG App Delivery is aware of this project as well as TAG Network. It's one of a few API management and schema projects we've heard about recently in our TAG. Relatedly, there are a couple efforts to create a WG to help these projects build awareness and support each other, see https://github.com/cncf/tag-app-delivery/issues/531 and https://github.com/cncf/tag-app-delivery/issues/448.

@joshgav - does TAG App Delivery have a recommendation?

Hi @amye - we posted at the same time. TAG App Delivery recommends CNCF accept this project in sandbox, yes!

Yaay!! congrats @akshayjshah and The entire team at Buf.

Thrift, Dubbo etc are not in the CNCF, they are Apache projects.

Thrift, Dubbo etc are not in the CNCF, they are Apache projects.

They're also listed on the CNCF landscape for RPC. I assumed that this meant that the projects were also CNCF-affiliated in some way, but perhaps that's not the case?

Regardless, apologies for any inaccuracy.

Hi TOC! We appreciate your consideration of Connect. Based on the recorded discussion of this application, it sounds like the TOC had two significant concerns:

1. First, the perception that Connect is primarily mobile- and web-focused. We do think that existing RPC solutions are especially atrocious in those environments, so we targeted our early efforts there. However, many adopters use Connect exclusively for backend TypeScript and Go services, and we're considering expanding to Python, Rust, C++, and/or Java this year. We touched on this point with TAG Network and TAG App Delivery, but we can clarify this application as well.
2. Second, the CNCF's overall concern that RPC isn't sufficiently cloud native. It sounded like that concern applies to gRPC, Connect, and possibly other projects as well. A discussion with TAG App Delivery in March was mentioned - is participating in that discussion the next step for us? Should we coordinate the details with TAG App Delivery?

Thanks again for your consideration! We're hoping to be ready for re-review in April.

However, many adopters use Connect exclusively for backend TypeScript and Go services

To add to this, as a member of the Connect team and an early adopter, we at PlanetScale have fully adopted Connect RPC to replace most of our internal usages of gRPC on the order of hundreds of thousands of requests per second up to the order of millions of RPCs per second all for internal communications. We use it between internal services, to interact with internal gRPC services, Connect services, and use our own wrappers for talking to etcd.

I would agree with @akshayjshah that a focus on mobile is more of a deficit for gRPC ecosystem so it's an easy target. For example, we at PlanetScale directly leverage the ability to use Connect RPC from a browser with a simple fetch() API rather than needing to run some extra gRPC gateway or shims to support that.

Personally, my opinion is that we utilize it in any case to replace our previous usages of grpc-go with the added benefits of having first-class browser support.

This fully supports our database-js database driver implementation which targets runtimes that can't use gRPC easily, for example. And we expect to develop more database drivers that speak over HTTP/3 in the future, which is enabled by Connect. https://planetscale.com/blog/faster-mysql-with-http3 Most of these uses cases are outside of browser and mobile and are targeting server <> server communications.

@amye I think we have sufficient information regarding this project to have it return to the queue for our April meeting.

Connect currently uses a CLA to Buf Technologies for contributions. Do you plan to transfer this CLA to the CNCF, or will you switch to a DCO?

Connect has the usual set of contributor/project management documentation. Note that the project already has a maintainer who does not work for the project sponsor.

Connect currently uses a CLA to Buf Technologies for contributions. Do you plan to transfer this CLA to the CNCF, or will you switch to a DCO?

We plan to transfer the CLA to the CNCF.

Connect has the usual set of contributor/project management documentation. Note that the project already has a maintainer who does not work for the project sponsor.

Of the 11 Connect maintainers, 3 don't work for the project sponsor:

• Matt Robenolt (@mattrobenolt)
• Michael Rebello (@rebello95)
• Eddie Seay (@eseay)

We're proud of how open Connect is, and we'd love to welcome new maintainers!

@justincormack Thanks for pointing out my misunderstanding of the CNCF landscape - I'd originally assumed that Apache projects can somehow also become CNCF-affiliated, but now I see that any project can submit themselves for inclusion in the landscape. I've updated the application to correctly categorize Thrift, Dubbo, and a few less-prominent RPC frameworks. Apologies for any confusion.

I see that both TAG-network and TAG-app-delivery recommends CNCF accept this project in sandbox, any strong feeling of which TAG it should fit best @akshayjshah @joshgav @nicholasjackson @leecalcote?

@joshgav curious if TAG-app-delivery currently has similar projects like Connect? I think TAG network has gRPC.

My vote is to accept this as a CNCF sandbox project, and we need to sort out which TAG is the best fit.

Thank you for your continued support, @linsun! I think everyone's in agreement that TAG Network is the right home for Connect:

• Speaking for the Connect maintainers, we'd prefer to be under TAG Network (alongside gRPC).
• TAG App Delivery also recommends that Connect work primarily with TAG Network.
• TAG Network's notes are slightly less formal, but we've presented to them three times: an overview of the project last year, another overview this year (after the change in TAG leadership), and a deep dive into wire protocols. After each presentation, my strong impression has been that TAG leadership feels that Connect fits best with Network. @nicholasjackson, please correct me if I've misrepresented our last few conversations!

TAGNetwork would very much welcome Connect, personally I think it is an amazing framework and I we are all very much impressed with the way that it works, and what the team has done to create it.

Thank you @akshayjshah and @nicholasjackson! Appreciate the thoughts, and my vote is also for TAG-network as a sandbox project, following the lead of gRPC.

TAG App Delivery agrees that TAG Network is the best primary TAG for this project cc: @dims @linsun