pgcat icon indicating copy to clipboard operation
pgcat copied to clipboard

Almost automatic shard detection

Open levkk opened this issue 3 years ago • 2 comments

Automatically route to the right shard based on:

  1. if Q (simple query), sharding_key = <value> or comment that says /* sharding_key = <value> */
  2. if P (prepared, extended protocol), a comment that says /* sharding_key = <value> */

The more advanced version of this is to parse the SQL and get it out of Q and B packets depending on protocol. That one will require no modifications to clients (Ruby, Go, etc).

levkk avatar Sep 12 '22 19:09 levkk

Isn't this at parity with what we have today in that the client has some responsibility for sending something to determine shard? This change is just trying to address the latency in the extra query?

zainkabani avatar Sep 13 '22 22:09 zainkabani

Isn't this at parity with what we have today in that the client has some responsibility for sending something to determine shard? This change is just trying to address the latency in the extra query?

I'm considering changing every decision the pooler makes to be automatic instead of manual. Having our own syntax is fun, but it does, as you said, add latency. The end goal for this project is to abstract all scaling decisions away from the client and "just work".

levkk avatar Sep 13 '22 23:09 levkk

Closing in favor of #293

levkk avatar Jan 25 '23 17:01 levkk