algebird
algebird copied to clipboard
twitter
Change type for rollup
Right now, cubing and rollup have the same type of keys: tuples or Options.
The problem is, for rollup, most of the possible values are uninhabited. For instance None, Some(_) would never happen. A better type would be:
(A, B) => Option[(A, Option[B])]
(A, B, C) => Option[(A, Option[(B, Option[C])])]
now, occasionally we chose looser types because using them is significantly easier, for instance in outerJoin in scalding we make the value type be: (Option[V1], Option[V2]) but (None, None) is impossible (we'd be better with a type that EitherOrBoth or something that has three values).
If we are going to change this, we should do it before we publish a new version (even though it does not break binary compatibility because the types that change are generated by a macro).
Note, scalding's TypedText type could flatten this key to read and write it without a problem (unless the values are Strings, since Option[String] is ambiguous.
/cc @sid-kap @ianoc
Yes. It is like an HList that may have a length from 0 to n, and that is not known at compile time, but given a position in the list, if present, the type is known.
I'm too lazy to think about this right this second, but it seems like there might be an infix type constructor notation that would make these nested options less awkward.
This makes sense.
As a bonus, this would allow for dealing with case classes with more than 22 values. Also, it would mean that the user doesn't have to see long strings of None,None,None.
Yeah, so if you do a right-associative infix type constructor it works:
type >>:[A,B] = (A,Option[B])
type StringIntDoubleRollup = Option[String >>: Int >>: Double]
val x: StringIntDoubleRollup = Some(("foo", Some((1, Some(1.0)))))
Thing I'd wonder about this: is will all use cases just revert to converting to a tuple of options? Most use cases like writing out to thrift will be much more natural/easy with a struct of optional fields, we'd just end up immediately unpacking this alternate type right?
That's a cool trick, @avibryant. One wonders if not having TupleN rather than a similar right associative function and unapplies would circumvent some of the Tuple22 pain:
val myTup: Int ->: String ->: Int = 1 ->: "yo" -> 2
val x ->: y ->: z = myTup // unapply here
Of course, you could do the nesting in the first position rather than the second and not need the right associativity (and ensuing : ugliness).