mecab-python3 version

[polm]

Hello, I'm the maintainer of mecab-python3. I notice you have the version pegged to <1.0, is there a reason for that?

If possible it'd be great to have the latest version used here, along with pip-packaged dictionaries like unidic-lite and ipadic-py.

I'd be glad to help out with any required changes.

[taishi-i]

Hello, @polm

The reason for selecting versions of mecab-python3 is because the current version of transformers with cl-tohoku/bert-base-japanese-whole-word-masking is not available.

Toiro provides the text classification model based on transformers (3.0.2). So, I need to specify versions of mecab-python3 (<=0.996.5) to use it.

I will upgrade the version of mecab-python3 when the following changes are merged to the next version of transformers. https://github.com/huggingface/transformers/commit/48c6c6139fbb2881ef16ac5d8afb6287467bf66e

[polm]

OK, that makes sense, thanks for the clarification.