Query on Best Practices for Handling High Bandwidth on the Coordinator in Large Citus Clusters

[antonissmal]

Hello Citus Community,

I am currently designing a Citus cluster expected to manage over 100TB of data and handle millions of queries. Given the high traffic anticipated, I am concerned about the potential for the coordinator to become a bottleneck, despite the scalability improvements in Citus 11.

According to the Citus documentation, the coordinator is primarily responsible for storing metadata and final aggregations, and while it's possible to add another coordinator, it doesn't mention handling multiple primary coordinators.

With this setup:

• Is there a recommended approach or best practices for managing high bandwidth impacts on the coordinator?
• Could you provide insights or examples of how other large-scale deployments have optimized coordinator performance under similar conditions?

Any advice or guidance would be greatly appreciated as we aim to optimize our architecture for high performance and reliability.

Thank you!