hsmecab icon indicating copy to clipboard operation
hsmecab copied to clipboard

`Stack space overflow' with a large input.

Open yuttie opened this issue 12 years ago • 2 comments

When parseToNode is applied to a large input text, it ends up with `Stack space overflow' error.

I made a sample program:

module Main where

import qualified Text.MeCab as MeCab

main :: IO ()
main = do
    m <- MeCab.new []
    let s = concat $ replicate 10000 "にわにはにわにわとりがいる。"
    ns <- MeCab.parseToNode m s
    print $ length ns

and compiled and ran it as follows:

% ghc -Wall test_mecab.hs
[1 of 1] Compiling Main             ( test_mecab.hs, test_mecab.o )
Linking test_mecab ...
% ./test_mecab
Stack space overflow: current size 8388608 bytes.
Use `+RTS -Ksize -RTS' to increase it.

This problem doesn't happen when I use `runhaskell' to execute it:

% runhaskell ./test_mecab.hs
70002

yuttie avatar Sep 11 '12 09:09 yuttie

I'm using GHC 7.4.2 and the version 0.3.0 of this mecab package.

yuttie avatar Sep 11 '12 10:09 yuttie

This is very likely due to using a String. If you use Text from the text package the problem goes away.

agrafix avatar Feb 19 '17 13:02 agrafix