source为较长的纯数字时，导致cpu耗尽，严重影响写入性能

[okailiang]

当PinyinTokenizer.java中处理source为较长的连续数字，如26677,28887,277029295,24464,23771,23488,26618,7,16815,20893,20310,26907,20616,165323040,17698,18550,9,22148,218415058,10194,18890,15645,18296,9183,8170,8034,8326,14469,185,4310,6888,5562,5057,4346,3170,2850,2707,842165,1094101,9843734,9843368,983055,1093008,983691,1092646,983320,1092279,98247,999546,981208,999180,981840,998815,981470,998452,980886,998085,980729,997180,989358,979990,996992,979620,996627,979248,996262,978877,995897,978510,995531,978140,995775,994797,977406,994430,976036,994063,976669,993698,976298,993926,992965,975556,992598,975187,992231,974816,991865,974446,991498,974075,991132,973705,990767,973334,990401,972965,989596,989668,972225,989302,971855,988935,971488,988570,971052,988206,970746,987840,970324 。

调用PinyinAlphabetTokenizer#walk时，时间复杂度放大，导致线程长时间RUNNABLE，堆栈信息为： "elasticsearch[es-0][write][T#1]" #85 daemon prio=5 os_prio=0 tid=0x00007f5eac01f000 nid=0x242 runnable [0x00007f5e53835000] java.lang.Thread.State: RUNNABLE at org.elasticsearch.index.analysis.PinyinAlphabetTokenizer.parse(PinyinAlphabetTokenizer.java:63) at org.elasticsearch.index.analysis.PinyinAlphabetTokenizer.walk(PinyinAlphabetTokenizer.java:45) at org.elasticsearch.index.analysis.PinyinTokenFilter.parseBuff(PinyinTokenFilter.java:279) at org.elasticsearch.index.analysis.PinyinTokenFilter.readTerm(PinyinTokenFilter.java:161) at org.elasticsearch.index.analysis.PinyinTokenFilter.incrementToken(PinyinTokenFilter.java:86)

应该在PinyinAlphabetTokenizer#walk 和 PinyinAlphabetTokenizer#parse 判断输入串和字串是否为纯数字，纯数字则直接返回：

public static List<String> walk(String text) {
        int maxLength=6;
        text = text.toLowerCase();
        LinkedList<String> candidates=new LinkedList<>();
        // text中不含有字母直接返回
        if (isNumeric(text)) {
            candidates.add(text);
            return candidates;
        }
       ... 省略...
     }

private static void parse(LinkedList<String> candidates, StringBuffer buffer,Boolean last) {
        String subText = buffer.toString();
        if (isNumeric(subText)) {
            return;
        }
         ... 省略...
 }

public static boolean isNumeric(CharSequence cs) {
        if (cs == null || cs.length() == 0) {
            return false;
        } else {
            int sz = cs.length();
            for(int i = 0; i < sz; ++i) {
                if (!Character.isDigit(cs.charAt(i))) {
                    return false;
                }
            }
            return true;
        }
    }

[KinYu668]

问题解决了吗？我这边也是遇到类似的问题，一个文档内容包含了base64转码，会有一大堆字符串。占用Bulk线程，并且发现有部分task running状态持续了8个小时。

[okailiang]

通过上面的方法修改源码解决了，你试下