impact-graph
impact-graph copied to clipboard
Implement Database Connection Pooling
After having a rough time in our backend and after noticing some connection latency to our Database due to many reasons some of them might be:
- Poor DB performance
- Poor connection latency
- Poor Query Optimisation
- No Timeout set to DB Connection sessions to be cleared
- No replication on the DB connection backend.
Although I'm not qualified to assume how (Or if) our queries can be optimized more to fix the problem at the moment, but from my findings I noticed that we can do some measures to prevent latency from happening (Or at least decrease it by a lot)
NOTE:
If I am assuming right:
-
Latency in the Database
--> 2.Causes Disconnections In the backend
--> 3.Causes the frontend errors that we see
Connection pooling documentation for Digital Ocean: https://docs.digitalocean.com/glossary/connection-pool Connection pooling for Typeorm: https://github.com/brianc/node-postgres/tree/master/packages/pg-pool
Note in TypeORM the connection pooling is supported out-of-the-box as I've read here: https://stackoverflow.com/questions/67414360/connection-pooling-in-typeorm-with-postgressql
CC: @jainkrati @laurenluz @mohammadranjbarz @aminlatifi @geleeroyale @Rolazo
Note that this is different from supporting READ and READ/WRITE replicas, the issue that was opened here: https://github.com/Giveth/impact-graph/issues/1016
@mhmdksh Thanks for the report, you say we don't need to do anything to utilize the connection pool and it's already utilized? otherwise, what would be the next step in your view?
@mhmdksh Thanks for the report, you say we don't need to do anything to utilize the connection pool and it's already utilized? otherwise, what would be the next step in your view?
Yes @aminlatifi As I was researching and like I said this is supported natively when using TYPEORM (Thanks guys). We were able to support this on both Production and Staging with Digital Ocean and AWS.
However there was one small configuration field added later by in this commit https://github.com/Giveth/impact-graph/commit/9a70d1162e1d77cd4c1536944c3262b85ceb658f that will help us in modifying the pooling config. Thanks to @mohammadranjbarz
We are monitoring the performance, but I think this issue can be closed now
This issue is on UAT column How can I test it ?
@mhmdksh pls work with @maryjaf to close this as its already in production and we can close after testing.
I've checked with @mhmdksh and we can close this issue.