split-lang
split-lang copied to clipboard
✨ Split text by languages (e.g. 你喜欢看アニメ吗 -> 你喜欢看 | アニメ | 吗) for NLP tasks (e.g. parse, TTS). Powered by fasttext and langua
# Motivation - Current test text examples focus on Chinese, Japanese, Korean and English - Since I can only speak Chinese, Japanese and English, if you your use cases meet...
金六福双旦大惠战,年终献礼真情回馈!黄金白菜价,仅售275/克,铂金全场8.5折,硬金满一千减三百元,满二千减六百元,依此类推,买的多,减的多!钻石吊坠劲爆价只要899元一件、10分钻戒只要1688元,65分钻戒只要9599元。旧金免工费并升值30元/克换购珠宝。买珠宝送玉器送银饰,更有足金貔貅和金鼠赠送!欢迎各位新老顾客进店选购!
# Bad Examples - Japanese - `。「` should be separated, `。` belongs to previous sentence while `「` belongs to next sentence - `。」` should belongs to previous sentence # Solution...