scalding
scalding copied to clipboard
Consistently call validateTap in all src.toIterator implement
All toIterator impls should consistently validate taps in order to be able to rely on the iterator completeness. Compare com.twitter.scalding.Mappable#toIterator and com.twitter.scalding.commons.source.LzoCodec#toIterator, etc.
cc @isnotinvain the whole validation thing is racy also. I'm not sure validateTap is the full solution.
It may be that we want make it easier for the the taps to validate as they read so they don't return partial results.