Document options for load balancing / best practices for connecting to a cluster

[danielmewes]

trafficstars

@timjrobinson wrote in https://github.com/rethinkdb/rethinkdb/issues/2956:

I can't find any documentation on how to properly run a cluster of 2+ nodes with rethinkdb. Should I just set them up and make clients connect to the first server? Or should I use a loadbalancer such as HAProxy or nginx in front of the cluster and go through that? If I should be using a loadbalancer what settings should I be specifying for maximum performance?

It would be good to have a doc that explains even the basics of how this should be setup. If you could point me to any docs related to this it would be much appreciated.

[coffeemug]

:+1: /cc @chipotle

[coffeemug]

Note, https://github.com/rethinkdb/rethinkdb/issues/281 is relevant here; we'll have to change the doc once that's implemented.

[chipotle]

This seems like it'd be a good document -- I'm wondering whether it should be a standalone document or an expansion/rewrite of the "Create a cluster on system startup" document we already have. Or both. To me, "cluster" implies "more than one" to begin with; we could rename the document "Starting RethinkDB," reorganize what's there a little, and include a section on what parameters you pass to each server to make them into a cluster. (AFAIK, there's not actually a lot to that, which is why it seems like it might be better as just a section in a longer document -- but I could wrong.)

On Wed, Aug 27, 2014 at 2:10 AM, Slava Akhmechet [email protected] wrote:

Note, rethinkdb/rethinkdb#281 https://github.com/rethinkdb/rethinkdb/issues/281 is relevant here; we'll have to change the doc once that's implemented.

— Reply to this email directly or view it on GitHub https://github.com/rethinkdb/docs/issues/484#issuecomment-53544404.

[coffeemug]

I think we should have a standalone "Cluster setup best practices" document (or something like that). It's non-trivial to a lot of people, and have a nice document that explains the dos and don't would help a lot to reassure people that their setup is good.

[danielmewes]

we could rename the document "Starting RethinkDB," reorganize what's there a little,

:+1: something we should definitely do.

In addition I think we should have a separate document that's just about cluster configuration as @coffeemug said. The document should not just focus on the configuration options, but give a higher level overview addressing questions such as the one which node(s) clients should connect to, and have links to the more detailed documents on how to secure a cluster, on what to do in case of a node failure etc.

In fact we could consider structuring it as a FAQ. What do you think? (@chipotle)

We should also adapt http://rethinkdb.com/docs/start-a-cluster/ a little bit. It's a nice tutorial for people who want to play around with RethinkDB, but has little significance for production deployments. It currently links to the "Create a cluster for system startup" as a resource for how to set up a production cluster. At the very least we should replace that link with a link to the new clustering overview document.

[hellodk]

@danielmewes we have two client machines, how do we connect both of them using proxy server? As you said earlier: "To start a RethinkDB proxy on the client: rethinkdb proxy -j -j ..." only of the clients can connect in this way, since the ports will already be in use.

[danielmewes]

As mentioned on the mailing list:

Try passing‎ '-o 1' to rethinkdb proxy. That will shift the ports that it's using by an offset of 1.