piccolo
piccolo copied to clipboard
Support for read replica for read only transactions
Enhancement request: Ability to leverage read replica for read only transactions to improve DB performance.
Some rough thoughts on the implementation:
- Create an Aurora Postgres cluster with one writer and one reader node.
- The config should specify
db identifier
and maybe optionalenable read replica
flag. - It connects to AWS, looks up the RDS cluster, identifies the endpoints for writer and reader nodes and creates respective connection pools.
- If a transaction is marked readonly, it uses a connection from the read replica else the writer node.
Not sure if this is feasible, but given lot of production projects now use read replica (atleast for high availability failover), DB performance could be improved by leveraging the read replica.
We have something a bit like this, for Postgres. I you look here at the extra_nodes
argument:
https://piccolo-orm.readthedocs.io/en/latest/piccolo/engines/postgres_engine.html#source
To specify it you do this in your piccolo_conf.py
:
DB = PostgresEngine(
config={'database': 'main_db'},
extra_nodes={
'read_replica_1': PostgresEngine(
config={
'database': 'main_db',
host: 'read_replicate.my_db.com'
},
extensions=()
)
}
)
It allows you to do things like this:
await MyTable.select().run(node='read_replica_1')
We should probably add a separate section in the docs for it.
There's still room for improvement. What are the main enhancements you think we should add to it?