node-chardet

node-chardet copied to clipboard

Hello there!

I am trying to use chardet while parsing big (20-30k+ rows) csv files. When i am calling chardet.detect() it returns result for first 10-14 elements and then main thread just dies and nothing happenning. Same thing in web-worker context (still inside electron).

My code (it is working in electron app in browser main thread context):

const chardet = require("chardet");

hugeArrayOfRowsParsedFromCSVFile.forEach((row, index) => {
    if (index > 100 && index < 200) {
        console.log( chardet.detect( row[0] ) ); //  row[0] is always a String
    }
}

Can the problem be in bad implemented electron's API (require)? I din't yet try chardet in non-electron browser context.

Ok... i've found out that value to detect was not always a String - sometimes it is a Number, sometimes undefined... But why there is no error message then? I've got once an error something like: "Argument must be a ArrayBuffer or a String"

Sorry for the 11 months delay 😬

Would you be able to share a test data for this case?

I've run into this as well. It's easy to replicate in the Node repl:

$ node
const chardet = require('chardet');
chardet.detect(123);

This should be fixed now via #90