elasticsearch-jdbc
elasticsearch-jdbc copied to clipboard
Too many thread pools created
线程数不会销毁,我用的版本是2.3.4.0
@githubyandou524 please in english.
I have met a problem, when i use the JdbcImporter for a while , too many threads are created(about 2000 in 10 hours).
I have encounter same issue while doing scheduled imports, i believe issue is related with this line in JDBCImporter.java: this.executorService = Executors.newFixedThreadPool(settings.getAsInt("concurrency", 1)); where we create new pool in prepare function. I was able to create simple fix by creating fixed size pool when executorService variable is declared. I'm testing solution at moment to clear that issue is in prepare function. Will be happy to provide more robust patch if i determine that this is root cause of issue. I'm testing 2.3.4.1 version. More completed solution would be checking why this threads stay in WAIT state and not closing properly.
Thank you for your analysis and your effort. The issue will be fixed in upcoming JDBC importer releases.
@jprante Can this issue have fixed in latest version? when i pull the master branch and run it in my local, the thread can not be finalize,and the number of thread always increase, I use jconsole to inspect this issue
@jprante I think I have found the reason of this issue. the JDBCImporter is a latest Object when program schedule. this will cause the executorService is latest too. may be should shutdown executorService after scheduled. like this:
finally {
try {
if (context != null) {
context.shutdown();
context = null;
}
if (executorService != null) {
executorService.shutdown();
executorService = null;
}
} catch (Throwable e) {
logger.warn(e.getMessage(), e);
}
}