elephant-bird icon indicating copy to clipboard operation
elephant-bird copied to clipboard

Lucene merge multiple indexes into single using HdfsMergeTool Failing

Open manoharballia opened this issue 8 years ago • 0 comments

I am trying to merge multiple solr collections into single using provided tool HdfsMergeTool. But i get the bellow error: Used command : hadoop jar HdfMergeSolr.jar /work/slr_merged_1_and_2 2 /solr/collection_1 /solr/collection_2

Error Log: 16/09/21 09:28:01 INFO mapreduce.Job: Job job_1469767030336_14546 running in uber mode : false 16/09/21 09:28:01 INFO mapreduce.Job: map 0% reduce 0% 16/09/21 09:28:09 INFO mapreduce.Job: Task Id : attempt_1469767030336_14546_m_000000_0, Status : FAILED Error: org.apache.lucene.index.LogByteSizeMergePolicy.setUseCompoundFile(Z)V 16/09/21 09:28:17 INFO mapreduce.Job: Task Id : attempt_1469767030336_14546_m_000000_1, Status : FAILED Error: org.apache.lucene.index.LogByteSizeMergePolicy.setUseCompoundFile(Z)V 16/09/21 09:28:21 INFO mapreduce.Job: Task Id : attempt_1469767030336_14546_m_000000_2, Status : FAILED Error: org.apache.lucene.index.LogByteSizeMergePolicy.setUseCompoundFile(Z)V 16/09/21 09:28:27 INFO mapreduce.Job: map 100% reduce 0% 16/09/21 09:28:27 INFO mapreduce.Job: Job job_1469767030336_14546 failed with state FAILED due to: Task failed task_1469767030336_14546_m_000000 Job failed as tasks failed. failedMaps:1 failedReduces:0

16/09/21 09:28:27 INFO mapreduce.Job: Counters: 11 Job Counters Failed map tasks=4 Launched map tasks=4 Other local map tasks=4 Total time spent by all maps in occupied slots (ms)=18101 Total time spent by all reduces in occupied slots (ms)=0 Total time spent by all map tasks (ms)=18101 Total vcore-seconds taken by all map tasks=18101 Total megabyte-seconds taken by all map tasks=74141696 Map-Reduce Framework CPU time spent (ms)=0 Physical memory (bytes) snapshot=0 Virtual memory (bytes) snapshot=0 Exception in thread "main" java.lang.RuntimeException: Merge tool exited with code: -1 at com.sears.MergeSolrIndex.HdfsMergeTool.main(HdfsMergeTool.java:105) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:606) at org.apache.hadoop.util.RunJar.run(RunJar.java:221) at org.apache.hadoop.util.RunJar.main(RunJar.java:136)

manoharballia avatar Sep 21 '16 13:09 manoharballia