magpie
magpie copied to clipboard
magpie 2.X - Support erasure coding option to terasort
In branch magpie20_erasureencodingterasort, will not work until later
https://issues.apache.org/jira/browse/HDFS-7661
although it's possible to hack teragen/terasort to work around this.
JIRA says may not be fixed for 3.0 release.
also, add to testsuite erasure code option for testing
confirmed
--- a/hadoop-mapreduce-project/hadoop-mapreduce-examples/src/main/java/org/apache/hadoop/examples/terasort/TeraOutputFormat.java
+++ b/hadoop-mapreduce-project/hadoop-mapreduce-examples/src/main/java/org/apache/hadoop/examples/terasort/TeraOutputFormat.java
@@ -77,7 +77,7 @@ public synchronized void write(Text key,
public void close(TaskAttemptContext context) throws IOException {
if (finalSync) {
- out.hsync();
+ // out.hsync();
}
out.close();
}
Fixes teragen to work for the time being.