stacks-blockchain-api
stacks-blockchain-api copied to clipboard
feat: ensure events are inserted into the raw event request table
Issue found by @csgui where the event_observer_requests table can be missing events even if the rest of the event ingestion was successful.
This appears to be because the function for inserting into event_observer_requests was ran after the event server http request has been completed. So when a large block was received, it may have been ingested into the regular tables, but could run into an OOM error on the event_observer_requests insert.
This PR fixes the issue by ensuring the http request is not completed until insert into event_observer_requests is completed. Also the amount of JSON parsing/stringifying has been reduced in an attempt to reduce likelihood of an OOM when receiving a large /new_block event payload.
Vercel deployment URL: https://stacks-blockchain-i7qr60ei6-blockstack.vercel.app :rocket:
Codecov Report
Attention: Patch coverage is 85.18519% with 4 lines in your changes are missing coverage. Please review.
Project coverage is 72.02%. Comparing base (
f522d79) to head (e5299b9). Report is 49 commits behind head on master.
:exclamation: Current head e5299b9 differs from pull request most recent head cb4016d
Please upload reports for the commit cb4016d to get more accurate results.
| Files | Patch % | Lines |
|---|---|---|
| src/event-stream/event-server.ts | 86.36% | 3 Missing :warning: |
| src/datastore/pg-write-store.ts | 80.00% | 1 Missing :warning: |
Additional details and impacted files
@@ Coverage Diff @@
## master #1925 +/- ##
==========================================
- Coverage 72.07% 72.02% -0.06%
==========================================
Files 93 93
Lines 12026 12027 +1
Branches 2654 2654
==========================================
- Hits 8668 8662 -6
- Misses 3202 3209 +7
Partials 156 156
:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.
@zone117x Just validated this fix against an archive synced from genesis. Please check the conversation on this devops issue for more context.
Blocks containing a large number of transactions are now included in the archive file.
For example, block 137843, which was the root cause of this issue, is now present in the archive file after the fix. Therefore, for a block to be present in the archive file, it must have also been present in the event_observer_requests table.
Fix LGTM.
Awesome, thanks for the thorough testing @csgui!
:tada: This PR is included in version 7.11.0-beta.8 :tada:
The release is available on:
Your semantic-release bot :package::rocket:
:tada: This PR is included in version 7.11.0 :tada:
The release is available on:
Your semantic-release bot :package::rocket: