wikihadoop
wikihadoop copied to clipboard
Non-uniform progress report
The percentage of the progress usually does't go up until 100%, even when all the data is processed.
The cause of this problem is that the current implementation calculate the progress in a way that the percentage is 100% when the file pointer reaches to the end of the split being processed. However, this is usually incorrect because the portion of the split after the last occurrence of is just skipped.