ClassNotFoundException: TimeBasedIndexNameBuilder

[pricecarl]

Hey,

First of all thank you for creating this. If i can get it working it will be a great help. The issue I have on restarting flume is this:

2:25:55.970 PM  ERROR   org.apache.flume.node.AbstractConfigurationProvider 
Sink enrichedEpoESSink has been removed due to an error during configuration
java.lang.RuntimeException: java.lang.ClassNotFoundException: TimeBasedIndexNameBuilder
at com.google.common.base.Throwables.propagate(Throwables.java:156)
at com.frontier45.flume.sink.elasticsearch2.ElasticSearchSink.configure(ElasticSearchSink.java:309)
at org.apache.flume.conf.Configurables.configure(Configurables.java:41)
at org.apache.flume.node.AbstractConfigurationProvider.loadSinks(AbstractConfigurationProvider.java:418)
at org.apache.flume.node.AbstractConfigurationProvider.getConfiguration(AbstractConfigurationProvider.java:103)
at org.apache.flume.node.PollingPropertiesFileConfigurationProvider$FileWatcherRunnable.run(PollingPropertiesFileConfigurationProvider.java:140)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:304)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:178)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:744)
Caused by: java.lang.ClassNotFoundException: TimeBasedIndexNameBuilder
at java.net.URLClassLoader$1.run(URLClassLoader.java:366)
at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
at java.lang.ClassLoader.loadClass(ClassLoader.java:425)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308)
at java.lang.ClassLoader.loadClass(ClassLoader.java:358)
at java.lang.Class.forName0(Native Method)
at java.lang.Class.forName(Class.java:190)
at com.frontier45.flume.sink.elasticsearch2.ElasticSearchSink.configure(ElasticSearchSink.java:303)
... 11 more

Any ideas what I'm doing wrong please?

[lucidfrontier45]

Hi, thanks for reporting. I fixed bug in ElasticSearchSinkConstants.java. Please try again.

[pricecarl]

Hey,

New error:

ERROR   org.apache.flume.lifecycle.LifecycleSupervisor  
Unable to start SinkRunner: { policy:org.apache.flume.sink.DefaultSinkProcessor@38cdca5b counterGroup:{ name:null counters:{} } } - Exception follows.
java.lang.IncompatibleClassChangeError: Found interface org.elasticsearch.common.settings.Settings, but class was expected
at com.frontier45.flume.sink.elasticsearch2.client.ElasticSearchTransportClient.openClient(ElasticSearchTransportClient.java:195)
at com.frontier45.flume.sink.elasticsearch2.client.ElasticSearchTransportClient.<init>(ElasticSearchTransportClient.java:78)
at com.frontier45.flume.sink.elasticsearch2.client.ElasticSearchClientFactory.getClient(ElasticSearchClientFactory.java:43)
at com.frontier45.flume.sink.elasticsearch2.ElasticSearchSink.start(ElasticSearchSink.java:337)
at org.apache.flume.sink.DefaultSinkProcessor.start(DefaultSinkProcessor.java:46)
at org.apache.flume.SinkRunner.start(SinkRunner.java:79)
at org.apache.flume.lifecycle.LifecycleSupervisor$MonitorRunnable.run(LifecycleSupervisor.java:251)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:304)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:178)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:744)

[lucidfrontier45]

Are you sure the latest elasticsearch2.0 libs are in your flume classpath?

[sobanik]

Hi, I have followed Your instructions and I'm receiving an errors like this bellow.

I have checked that my ElasticSearch is up and running and there is a ElasticSearch node available. I can access Kibana and Sense so I think that I need to configure something else as well.

I have added this new.jars files from Elastic as well as your elasticsearch-sink2-*.jar's to flume/lib/ directory.

What I'm missing?

015-12-08 10:47:32,663 (SinkRunner-PollingRunner-DefaultSinkProcessor) [ERROR - com.frontier45.flume.sink.elasticsearch2.ElasticSearchSink.process(ElasticSearchSink.java:201)] Failed to commit transaction. Transaction rolled back. NoNodeAvailableException[None of the configured nodes are available: []] at org.elasticsearch.client.transport.TransportClientNodesService.ensureNodesAreAvailable(TransportClientNodesService.java:280) at org.elasticsearch.client.transport.TransportClientNodesService.execute(TransportClientNodesService.java:197) at org.elasticsearch.client.transport.support.TransportProxyClient.execute(TransportProxyClient.java:55) at org.elasticsearch.client.transport.TransportClient.doExecute(TransportClient.java:272) at org.elasticsearch.client.support.AbstractClient.execute(AbstractClient.java:347) at org.elasticsearch.action.ActionRequestBuilder.execute(ActionRequestBuilder.java:85) at org.elasticsearch.action.ActionRequestBuilder.execute(ActionRequestBuilder.java:59) at com.frontier45.flume.sink.elasticsearch2.client.ElasticSearchTransportClient.execute(ElasticSearchTransportClient.java:178) at com.frontier45.flume.sink.elasticsearch2.ElasticSearchSink.process(ElasticSearchSink.java:185) at org.apache.flume.sink.DefaultSinkProcessor.process(DefaultSinkProcessor.java:68) at org.apache.flume.SinkRunner$PollingRunner.run(SinkRunner.java:147) at java.lang.Thread.run(Thread.java:745)

[lucidfrontier45]

what did you write to flume.conf? elasticsearch hosts must be specified there with the same options as official elasticsearch sink.

[lindzh]

log.sinks.elastic.type = com.frontier45.flume.sink.elasticsearch2.ElasticSearchSink log.sinks.elastic.hostNames = 192.168.2.220:9200,192.168.2.221:9200 log.sinks.elastic.indexName = aidaojia log.sinks.elastic.indexType = biz log.sinks.elastic.clusterName = simple log.sinks.elastic.indexNameBuilder = com.frontier45.flume.sink.elasticsearch2.TimeBasedIndexNameBuilder log.sinks.elastic.batchSize = 500 log.sinks.elastic.ttl = 5d log.sinks.elastic.channel = logChannel

[lucidfrontier45]

@lindzh what is your error?

[lindzh]

@lucidfrontier45 I fixed this problem with specified indexNameBuilder,trying to give a possible way to fix this issue without change the code

[jainsourabh2]

@lindzh try with 9300 port