tracker-radar-detector icon indicating copy to clipboard operation
tracker-radar-detector copied to clipboard

ERR_STRING_TOO_LONG for huge JSON file

Open mainzelM opened this issue 3 years ago • 0 comments

I did my own crawl with the tracker-radar-collector and tried to apply the detector on the result. This gave me the following error:

Error: Cannot create a string longer than 0x1fffffe8 characters
    at Buffer.utf8Slice (<anonymous>)
    at Object.slice (node:buffer:592:37)
    at Buffer.toString (node:buffer:810:14)
    at readFileHandle (node:internal/fs/promises:371:36)
    at async JSONFileDataReader.iterator (/somewhere/tracker-radar-detector/src/trackers/helpers/readers.js:31:19)
    at async processCrawl (/somewhere/tracker-radar-detector/src/trackers/process-crawl.js:48:26) {
  code: 'ERR_STRING_TOO_LONG'

I added some debug output and guess that the problem is the JSON file created for www.huya.com: it has a size of about 530MB. Is this a known issue? Do you need more information to reproduce this problem?

mainzelM avatar Jul 29 '21 15:07 mainzelM