scalding
scalding copied to clipboard
twitter
TypedText defaults to converting unparsed numbers to 0
A user has a case class wrapping a Long. The tsv had a string that could not be parsed as long, and the use got 0L for the Long.
I think we should throw in this case.
One interesting idea would be to have Try[Long] be the way to specify you want to catch exceptions in parsing. @ianoc has done similar tricks in some macros which I think worked out well.
Another thing we might think about is to allow the last parameter to be a List[String] to catch uneven tsvs which may sometimes have extra columns. Don't know how big of a deal this is.
We may not want to use cascading's parsing for this since with TypedText we have a macro that could directly generate the function of String => Try[T].
Try[Long] is a cool idea. How/how often do people hit this problem? What would a user do with a field that's Try[T], log it? Would it make more sense to have an ADT Parsable[T] that's like Parsed[T] xor Unparsable[String] to enable the user to deal with it?
Yeah, I like that. Or we could use Either[String, T] directly?
On Mon, Nov 28, 2016 at 08:45 Ben Pence [email protected] wrote:
Try[Long] is a cool idea. How/how often do people hit this problem? What would a user do with a field that's Try[T], log it? Would it make more sense to have an ADT Parsable[T] that's like Parsed[T] xor Unparsable[String] to enable the user to deal with it?
— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/twitter/scalding/issues/1624#issuecomment-263356762, or mute the thread https://github.com/notifications/unsubscribe-auth/AAEJdit49XwuS3oSxRzvXT1Y9UWTLTH-ks5rCyFLgaJpZM4K7FLk .
I thought about Either, but I'm not sure that would communicate to the consumer what's going on and it would commit us to using a general type for a specific purpose. Perhaps providing a Parsable#toEither would make it easier to use.