impact-graph icon indicating copy to clipboard operation
impact-graph copied to clipboard

Implement Database Connection Pooling

Open mhmdksh opened this issue 9 months ago • 2 comments

After having a rough time in our backend and after noticing some connection latency to our Database due to many reasons some of them might be:

  1. Poor DB performance
  2. Poor connection latency
  3. Poor Query Optimisation
  4. No Timeout set to DB Connection sessions to be cleared
  5. No replication on the DB connection backend.

Although I'm not qualified to assume how (Or if) our queries can be optimized more to fix the problem at the moment, but from my findings I noticed that we can do some measures to prevent latency from happening (Or at least decrease it by a lot)

NOTE:

If I am assuming right:

  1. Latency in the Database --> 2. Causes Disconnections In the backend --> 3. Causes the frontend errors that we see

Connection pooling documentation for Digital Ocean: https://docs.digitalocean.com/glossary/connection-pool Connection pooling for Typeorm: https://github.com/brianc/node-postgres/tree/master/packages/pg-pool

Note in TypeORM the connection pooling is supported out-of-the-box as I've read here: https://stackoverflow.com/questions/67414360/connection-pooling-in-typeorm-with-postgressql

CC: @jainkrati @laurenluz @mohammadranjbarz @aminlatifi @geleeroyale @Rolazo

Note that this is different from supporting READ and READ/WRITE replicas, the issue that was opened here: https://github.com/Giveth/impact-graph/issues/1016

mhmdksh avatar May 01 '24 08:05 mhmdksh

@mhmdksh Thanks for the report, you say we don't need to do anything to utilize the connection pool and it's already utilized? otherwise, what would be the next step in your view?

aminlatifi avatar May 01 '24 09:05 aminlatifi

@mhmdksh Thanks for the report, you say we don't need to do anything to utilize the connection pool and it's already utilized? otherwise, what would be the next step in your view?

Yes @aminlatifi As I was researching and like I said this is supported natively when using TYPEORM (Thanks guys). We were able to support this on both Production and Staging with Digital Ocean and AWS.

However there was one small configuration field added later by in this commit https://github.com/Giveth/impact-graph/commit/9a70d1162e1d77cd4c1536944c3262b85ceb658f that will help us in modifying the pooling config. Thanks to @mohammadranjbarz

We are monitoring the performance, but I think this issue can be closed now

mhmdksh avatar May 02 '24 13:05 mhmdksh

This issue is on UAT column How can I test it ?

maryjaf avatar May 08 '24 09:05 maryjaf

@mhmdksh pls work with @maryjaf to close this as its already in production and we can close after testing.

jainkrati avatar May 08 '24 12:05 jainkrati

I've checked with @mhmdksh and we can close this issue.

maryjaf avatar May 13 '24 10:05 maryjaf