impact-graph icon indicating copy to clipboard operation
impact-graph copied to clipboard

Published 20 hours ago • verified publisher icon Giveth

Too many calls for refresh project_donation_summary_view materialized view

Open aminlatifi opened this issue 9 months ago • 1 comments

There are plenty of such logs generated on the backend. Refreshing the materialized view is a heavy process that must not be done frequently. We must revise the policy of updating this materialized view, like debouncing!

query failed: |  
-- | --
  |   | error: error: canceling statement due to statement timeout |  
  |   |   |  
  |   | REFRESH MATERIALIZED VIEW project_donation_summary_view |  
  |   | query failed: |  
  |   | error: error: canceling statement due to statement timeout |  
  |   |   |  
  |   | REFRESH MATERIALIZED VIEW project_donation_summary_view |  
  |   | query failed:

aminlatifi avatar May 06 '24 12:05 aminlatifi

After @Rolazo set a time-limit for queries we can notice which queries are affected. Above query is constantly timing out. We will collect all the timed out queries in issues like this, but this seems to be the most unoptimized one

geleeroyale avatar May 06 '24 14:05 geleeroyale

Does this issue need to be tested on QA side? @CarlosQ96 @aminlatifi

maryjaf avatar May 08 '24 10:05 maryjaf

The merge didnt work, after disabling time our, refresh project_donation_summary_view materialized view was getting 6 minutes to be executed.

Rolazo avatar May 13 '24 18:05 Rolazo

@CarlosQ96 pls guide @RamRamez for this

jainkrati avatar May 29 '24 12:05 jainkrati

project_donation_summary_view is completely removed.

@maryjaf please test and verify

  • [x] sumDonationValueUsd and countUniqueDonors should be shown correctly for each project.
  • [ ] after each successful donation, sumDonationValueUsd and countUniqueDonors should be updated correctly

RamRamez avatar Jun 04 '24 13:06 RamRamez

  • after each successful donation, sumDonationValueUsd and countUniqueDonors should be updated correctly

after new donation the total donation has been updated correctly, but the number of donor isn't changed

-I've donated to this project with new donor -I haven't donated with this user before -I expect that the number of donor be changed from 41 to 42 but it isn't changed

maryjaf avatar Jun 09 '24 10:06 maryjaf

  • after each successful donation, sumDonationValueUsd and countUniqueDonors should be updated correctly

after new donation the total donation has been updated correctly, but the number of donor isn't changed

-I've donated to this project with new donor -I haven't donated with this user before -I expect that the number of donor be changed from 41 to 42 but it isn't changed

@maryjaf We had some other problems causing unexpected errors that are fixed now. Please test again.

RamRamez avatar Jun 12 '24 16:06 RamRamez

The problem related to number of contributor hasn't been fixed in Qf round donation tab

https://github.com/Giveth/impact-graph/issues/1607#issuecomment-2164669986

maryjaf avatar Jun 13 '24 08:06 maryjaf

The problem related to number of contributor hasn't been fixed in Qf round donation tab

#1607 (comment)

@maryjaf if the problem you mentioned here is resolved we can move it to done, we can track the other issue in its specific issue

RamRamez avatar Jun 13 '24 10:06 RamRamez

Yeah, I've tested and the problem related to number of unique donors has been resolved.

maryjaf avatar Jun 20 '24 07:06 maryjaf