mynlp icon indicating copy to clipboard operation
mynlp copied to clipboard

如何添加自己的词典?

Open zz1559152814 opened this issue 4 years ago • 1 comments

zz1559152814 avatar Aug 17 '20 08:08 zz1559152814

        MemCustomDictionary memCustomDictionary = new MemCustomDictionary();

        FluentLexerBuilder builder = Lexers.coreBuilder();

        builder.with(new CustomDictionaryPlugin(memCustomDictionary));

        Lexer tokenizer = builder.build();

        System.out.println(tokenizer);

        System.out.println(tokenizer.scan("欢迎来到松江临港科技城"));

        memCustomDictionary.addWord("临港科技城");
        memCustomDictionary.rebuild();

        System.out.println(tokenizer.scan("欢迎来到松江临港科技城"));

Output:

PipelineTokenizer

BestPathAlgorithm = ViterbiBestPathAlgorithm
CharNormalize = DefaultCharNormalize
WordTermCollector = SentenceCollector
WordSplitAlgorithm = CoreDictionarySplitAlgorithm,AtomSplitAlgorithm
WordpathProcessor = 
	CustomDictionaryProcessor

欢迎 来到 松江 临港 科技城
欢迎 来到 松江 临港科技城

jimichan avatar Aug 18 '20 02:08 jimichan