doublearray
doublearray copied to clipboard
stringToUtf8Bytes may return null
Fix TypeError: Cannot read properties of null (reading 'length')
.
I have not been able to reproduce the bug but it seems that stringToUtf8Bytes
may return null.
stringToUtf8Bytes returns null ↓ https://github.com/takuyaa/doublearray//blob/461f142db0e9111672e5528770546052a7a1ec30/doublearray.js#L685-L685
By the way, the error occurred while processing a string containing emoji like '😂' with kuromoji.
TypeError: Cannot read properties of null (reading 'length')
at DoubleArray.commonPrefixSearch ((...)/doublearray/doublearray.js:559:36)
at ViterbiBuilder.build ((...)/node_modules/kuromoji/src/viterbi/ViterbiBuilder.js:47:36)
at Tokenizer.getLattice ((...)/node_modules/kuromoji/src/Tokenizer.js:126:33)
at Tokenizer.tokenizeForSentence ((...)/node_modules/kuromoji/src/Tokenizer.js:81:24)
...
@takuyaa Today, I confirmed it happens when using 🍪
.