NOKUBI Takatsugu comments

Repositories
Issues
Comments

Results 5 comments of


                                            NOKUBI Takatsugu

LIME tokenizer for SentencePiece (or other tokenizer)

Unfortunately, the change didn't work well.

LIME tokenizer for SentencePiece (or other tokenizer)

I don't know another languages about non-space segmented language (maybe Chinese, Thai, ...), at least Japanese "word" is a little bit ambiguous consept. To segment to words , you need...

error while doing eval

It seems serialized model data size mismatch. If you trained before on the code before e61b1d4634c3c2b1f9c0c202aa493225e33f07ad commit, you need to re-train your data.

classes doesn't go beyond 9

I tried training your example and some other dataset (based on 20 newsgroups), it seems not work. At least, your example is reproducible, but it seems bad example because completely...

Relicensing to LGPL-2.1-or-later from LGPL-2.1-only

I agree to apply LGPL-2.1-or-later to my commits.