[Bug] Memory leak when using block handler without filter

[rattrap]

Bug report

For the following subgraph we needed a way to initialize some values (at block 0): https://github.com/Taraxa-project/taraxa-platform/tree/feature/add-subgraph-for-dpos/packages/dpos-subgraph

When we added a blockHandlers without a filter, it calls the handler for all blocks but the memory increases exponentially until the process is killed due to OOM.

The contents of the handler function doesn't matter. Also tested with an empty function.

Happens in Docker with v0.27.0, v0.28.0, v0.29.0, v0.30

https://user-images.githubusercontent.com/3051409/229186801-46673195-05fa-4f60-baa8-e2c636ee51e0.mp4

Relevant log output

No response

IPFS hash

No response

Subgraph name or link to explorer

No response

Some information to help us out

• [ ] Tick this box if this bug is caused by a regression found in the latest release.
• [ ] Tick this box if this bug is specific to the hosted service.
• [X] I have searched the issue tracker to make sure this issue is not a duplicate.

OS information

Linux

[leoyvens]

If you're writing entities in those handlers, it could be the caches are just getting too big. Try reducing GRAPH_ENTITY_CACHE_SIZE and GRAPH_STORE_WRITE_BATCH_SIZE. Also try writing no entities in the handler, and then seeing if this still reproduces.

[rattrap]

Hey, @leoyvens, Thanks for your answer. It's actually still reproducing even if the method is empty

[azf20]

hey @rattrap that is strange - you are running on Linux with latest Docker?

[github-actions[bot]]

Looks like this issue has been open for 6 months with no activity. Is it still relevant? If not, please remember to close it.