crawl icon indicating copy to clipboard operation
crawl copied to clipboard

CLI Allocation memory fail

Open Tilotiti opened this issue 10 years ago • 1 comments

Hi,

I'm trying to crawl a big website by using the CLI command

crawl -b http://www.***.com

Unfortunatly, after few minutes, the shell returns :

FATAL ERROR: CALL_AND_RETRY_2 Allocation failed - process out of memory
Abandon

I've found that you can increase the memory limit by using this option :

node --max-old-space-size=8192 my-node-script.js

So, How can I use this option with the CLI command ?

Thank you.

Tilotiti avatar May 13 '15 14:05 Tilotiti

I'm having the same problem right now and I can't get it to work. Has anyone figured this out?

jpearson311 avatar Jun 07 '16 13:06 jpearson311