crawl
crawl copied to clipboard
CLI Allocation memory fail
Hi,
I'm trying to crawl a big website by using the CLI command
crawl -b http://www.***.com
Unfortunatly, after few minutes, the shell returns :
FATAL ERROR: CALL_AND_RETRY_2 Allocation failed - process out of memory
Abandon
I've found that you can increase the memory limit by using this option :
node --max-old-space-size=8192 my-node-script.js
So, How can I use this option with the CLI command ?
Thank you.
I'm having the same problem right now and I can't get it to work. Has anyone figured this out?