api icon indicating copy to clipboard operation
api copied to clipboard

Memory Leak in `query.entriesPaged`

Open justraman opened this issue 1 year ago • 1 comments

  • I'm submitting a ...
  • [x] Bug report
  • [ ] Feature request
  • [ ] Support request
  • [ ] Other
  • What is the current behavior and expected behavior?

I've encountered an issue related to memory consumption while using the entriesPaged method. As I attempt to retrieve entries from the API in batches within a loop, I've noticed a steady increase in memory usage over time. This gradual buildup eventually leads to high memory consumption, crashes with FATAL ERROR: Ineffective mark-compacts near heap limit Allocation failed - JavaScript heap out of memory

Here is the code which lead to memory leak

   const BATCH_SIZE = 500;
	let lastKey = '';

	while (true) {
   	const query = await api.query.multiTokens.tokens.entriesPaged({
       	args: [],
       	pageSize: BATCH_SIZE,
       	startKey: lastKey,
   	});
   	
		if (query.length === 0) {
       	break;
   	}
   	for (const [key, value] of query) {
       	lastKey = key.toHex();
   	}
	}

please run it against rpc: wss://enjin-matrix-rpc-1.dwellir.com or you can also quickly reproduce this by running this on polkadot.js web.

  • Please tell us about your environment:

    • Version: 10.11.2

    • Environment: Node, Browser

      • [x] Node.js
      • [x] Browser
      • [ ] Other (limited support for other environments)
    • Language:

      • [x] JavaScript
      • [ ] TypeScript (include tsc --version)
      • [ ] Other

justraman avatar Feb 26 '24 19:02 justraman