stacks-blockchain-api icon indicating copy to clipboard operation
stacks-blockchain-api copied to clipboard

Ensure implicit SQL transaction is intact during parallel insertions

Open zone117x opened this issue 1 year ago • 0 comments

PR https://github.com/hirosystems/stacks-blockchain-api/pull/1818 introduced parallel insertions during block ingestion.

For reference here is the relevant code: https://github.com/hirosystems/stacks-blockchain-api/pull/1818/files#diff-6015cbd3e33ffb7064a4076b06232863ccf21ecc15a17f51dc4c4d223e31cd2bR264-R278

The write queries leverage implicit sql transaction handling. This is done via some magic involving AsyncLocalStorage: https://github.com/hirosystems/api-toolkit/blob/83acd010be5f1703e258d567c69c838fb6b05936/src/postgres/base-pg-store.ts#L51-L64

In the optimization PR, queries which previously used async/await in series are now added to a queue with callbacks, where think the async call stack is dropped, and the implicit sql transaction handling may stop working as expected. E.g. each of the callbacks invoked by the queue might be its own async context and therefore its own sql transaction.

This would be a critical bug because any issue with block ingestion would no longer rollback all changes made and corrupt the database.

zone117x avatar Jan 16 '24 12:01 zone117x