elephant-bird icon indicating copy to clipboard operation
elephant-bird copied to clipboard

Fix cast exception in DeprecatedOutputFormatWrapper

Open johnynek opened this issue 10 years ago • 0 comments

There is a cast: https://github.com/kevinweil/elephant-bird/blob/master/core/src/main/java/com/twitter/elephantbird/mapred/output/DeprecatedOutputFormatWrapper.java#L100

That can fail: org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:170) Caused by: java.lang.ClassCastException: org.apache.hadoop.mapred.Reporter$1 cannot be cast to org.apache.hadoop.mapreduce.StatusReporter at com.twitter.elephantbird.mapred.output.DeprecatedOutputFormatWrapper$RecordWriterWrapper.(DeprecatedOutputFormatWrapper.java:98) at com.twitter.elephantbird.mapred.output.DeprecatedOutputFormatWrapper.getRecordWriter(DeprecatedOutputFormatWrapper.java:84) at cascading.tap.hadoop.io.TapOutputCollector.initialize(TapOutputCollector.java:102) at cascading.tap.hadoop.io.TapOutputCollector.(TapOutputCollector.java:79) at cascading.tap.hadoop.io.TapOutputCollector.(TapOutputCollector.java:68) at cascading.tap.hadoop.io.HadoopTupleEntrySchemeCollector.makeCollector(HadoopTupleEntrySchemeCollector.java:57) at cascading.tap.hadoop.io.HadoopTupleEntrySchemeCollector.(HadoopTupleEntrySchemeCollector.java:49) at cascading.tap.hadoop.Hfs.openForWrite(Hfs.java:454) at cascading.tap.hadoop.Hfs.openForWrite(Hfs.java:91) at cascading.flow.hadoop.HadoopFlowProcess.openTrapForWrite(HadoopFlowProcess.java:289) at cascading.flow.stream.TrapHandler.getTrapCollector(TrapHandler.java:58) at cascading.flow.stream.TrapHandler.handleException(TrapHandler.java:131) at cascading.flow.stream.TrapHandler.handleException(TrapHandler.java:115) at cascading.flow.stream.ElementStage.handleException(ElementStage.java:134) at cascading.flow.stream.FunctionEachStage.receive(FunctionEachStage.java:103) at cascading.flow.stream.FunctionEachStage.receive(FunctionEachStage.java:39) at cascading.flow.stream.SourceStage.map(SourceStage.java:102) at cascading.flow.stream.SourceStage.run(SourceStage.java:58) at cascading.flow.hadoop.FlowMapper.run(FlowMapper.java:127) ... 7 more

johnynek avatar Feb 22 '14 02:02 johnynek