Seg fault with char_level=True

[stevenhillis]

I have a list of hypotheses and a list of references. Both have one element, a string. The hypothesis string has a length of 115233. The reference string has a length of 141575. I need to compute both WER and CER. When I compute WER with fastwer.score(hyps, refs), everything is normal. But, when I compute CER with fastwer.score(hyps, refs, char_level=True), my program pauses for an unexpected period of a few seconds, then prints "Segmentation fault", and then exits.

[shedar]

I had a similar issue. After looking at the source code, I guess it's out of memory error. Here it tries to allocate a vector of M*N size of int32 items. In your case, 115234 * 141575 *4 is over 60 gigabytes of memory just for this structure.