zimit
zimit copied to clipboard
openzim
"Invalid string length"
{"timestamp":"2024-10-30T07:15:08.970Z","logLevel":"info","context":"worker","message":"Worker done, all tasks complete","details":{"workerid":164}}
{"timestamp":"2024-10-30T07:15:08.970Z","logLevel":"info","context":"worker","message":"Worker done, all tasks complete","details":{"workerid":136}}
{"timestamp":"2024-10-30T07:15:08.970Z","logLevel":"info","context":"worker","message":"Worker done, all tasks complete","details":{"workerid":106}}
{"timestamp":"2024-10-30T07:15:08.970Z","logLevel":"info","context":"worker","message":"Worker done, all tasks complete","details":{"workerid":44}}
{"timestamp":"2024-10-30T07:15:08.970Z","logLevel":"info","context":"worker","message":"Worker done, all tasks complete","details":{"workerid":84}}
{"timestamp":"2024-10-30T07:15:08.970Z","logLevel":"info","context":"worker","message":"Worker done, all tasks complete","details":{"workerid":132}}
{"timestamp":"2024-10-30T07:15:08.970Z","logLevel":"info","context":"worker","message":"Worker done, all tasks complete","details":{"workerid":176}}
{"timestamp":"2024-10-30T07:15:08.970Z","logLevel":"info","context":"worker","message":"Worker done, all tasks complete","details":{"workerid":12}}
{"timestamp":"2024-10-30T07:15:08.970Z","logLevel":"info","context":"worker","message":"Worker done, all tasks complete","details":{"workerid":173}}
{"timestamp":"2024-10-30T07:15:08.970Z","logLevel":"info","context":"worker","message":"Worker done, all tasks complete","details":{"workerid":102}}
{"timestamp":"2024-10-30T07:15:08.970Z","logLevel":"info","context":"worker","message":"Worker done, all tasks complete","details":{"workerid":59}}
{"timestamp":"2024-10-30T07:15:08.970Z","logLevel":"info","context":"worker","message":"Worker done, all tasks complete","details":{"workerid":1}}
{"timestamp":"2024-10-30T07:15:08.970Z","logLevel":"info","context":"worker","message":"Worker done, all tasks complete","details":{"workerid":37}}
{"timestamp":"2024-10-30T07:15:08.970Z","logLevel":"info","context":"worker","message":"Worker done, all tasks complete","details":{"workerid":48}}
{"timestamp":"2024-10-30T07:15:08.970Z","logLevel":"info","context":"worker","message":"Worker done, all tasks complete","details":{"workerid":68}}
{"timestamp":"2024-10-30T07:15:08.970Z","logLevel":"info","context":"worker","message":"Worker done, all tasks complete","details":{"workerid":188}}
{"timestamp":"2024-10-30T07:15:08.970Z","logLevel":"info","context":"worker","message":"Worker done, all tasks complete","details":{"workerid":76}}
{"timestamp":"2024-10-30T07:15:08.970Z","logLevel":"info","context":"worker","message":"Worker done, all tasks complete","details":{"workerid":49}}
{"timestamp":"2024-10-30T07:15:08.970Z","logLevel":"info","context":"worker","message":"Worker done, all tasks complete","details":{"workerid":6}}
{"timestamp":"2024-10-30T07:15:08.970Z","logLevel":"info","context":"worker","message":"Worker done, all tasks complete","details":{"workerid":0}}
{"timestamp":"2024-10-30T07:15:08.970Z","logLevel":"info","context":"worker","message":"Worker done, all tasks complete","details":{"workerid":62}}
{"timestamp":"2024-10-30T07:15:08.970Z","logLevel":"info","context":"worker","message":"Worker done, all tasks complete","details":{"workerid":126}}
{"timestamp":"2024-10-30T07:15:08.970Z","logLevel":"info","context":"worker","message":"Worker done, all tasks complete","details":{"workerid":189}}
{"timestamp":"2024-10-30T07:15:08.970Z","logLevel":"info","context":"worker","message":"Worker done, all tasks complete","details":{"workerid":56}}
{"timestamp":"2024-10-30T07:15:09.051Z","logLevel":"info","context":"pageStatus","message":"Page Finished","details":{"loadState":3,"page":"https://ask.fm/basmaashraf6/answers/174374884757","workerid":28}}
{"timestamp":"2024-10-30T07:15:09.053Z","logLevel":"info","context":"worker","message":"Worker done, all tasks complete","details":{"workerid":28}}
{"timestamp":"2024-10-30T08:27:30.758Z","logLevel":"error","context":"general","message":"Crawl failed","details":{"type":"exception","message":"Invalid string length","stack":"RangeError: Invalid string length\n at writeBlockSequence (file:///app/node_modules/js-yaml/dist/js-yaml.mjs:3440:24)\n at writeNode (file:///app/node_modules/js-yaml/dist/js-yaml.mjs:3670:11)\n at writeBlockMapping (file:///app/node_modules/js-yaml/dist/js-yaml.mjs:3552:10)\n at writeNode (file:///app/node_modules/js-yaml/dist/js-yaml.mjs:3655:9)\n at writeBlockMapping (file:///app/node_modules/js-yaml/dist/js-yaml.mjs:3552:10)\n at writeNode (file:///app/node_modules/js-yaml/dist/js-yaml.mjs:3655:9)\n at Object.dump$1 [as dump] (file:///app/node_modules/js-yaml/dist/js-yaml.mjs:3781:7)\n at Crawler.serializeConfig (file:///app/dist/crawler.js:1658:26)\n at process.processTicksAndRejections (node:internal/process/task_queues:95:5)\n at async Crawler.crawl (file:///app/dist/crawler.js:927:9)"}}
{"timestamp":"2024-10-30T08:27:30.758Z","logLevel":"info","context":"general","message":"Exiting, Crawl status: failing","details":{}}
Traceback (most recent call last):
File "/usr/bin/zimit", line 8, in <module>
sys.exit(zimit.zimit())
^^^^^^^^^^^^^
File "/app/zimit/lib/python3.12/site-packages/zimit/zimit.py", line 688, in zimit
run(sys.argv[1:])
File "/app/zimit/lib/python3.12/site-packages/zimit/zimit.py", line 574, in run
raise subprocess.CalledProcessError(crawl.returncode, cmd_args)
subprocess.CalledProcessError: Command '['crawl', '--failOnFailedSeed', '--workers', '200', '--waitUntil', 'load', '--depth', '-1', '--timeout', '90', '--behaviors', 'autoplay,autofetch,siteSpecific', '--behaviorTimeout', '90', '--diskUtilization', '0', '--url', 'https://ask.fm', '--userAgentSuffix', '+Zimit', '--mobileDevice', 'Pixel 2', '--cwd', '/output/.tmpoj0i73ya']' returned non-zero exit status 9.
[zimit::2024-10-30 08:27:30,881] INFO:
[zimit::2024-10-30 08:27:30,881] INFO:----------
[zimit::2024-10-30 08:27:30,881] INFO:Cleanup, removing temp dir: /output/.tmpoj0i73ya
@ikreymer I think this issue is for you. Do you wanna me to open an issue in browsertrix-crawler repo?
