P. Oscar Boykin
P. Oscar Boykin
https://github.com/twitter/scalding/blob/398545991c63019714d80ac22de6494adfd798f2/scalding-core/src/main/scala/com/twitter/scalding/macros/impl/TypeDescriptorProviderImpl.scala If users override an implicit TypeDescriptor, use that if possible.
Note, inside of Job there is an implicit from `TraverseableOnce` to `Fields`: https://github.com/twitter/scalding/blob/develop/scalding-core/src/main/scala/com/twitter/scalding/FieldConversions.scala#L185 Note, `.isDefined` is a method on `Option` in scala, but not on `TraversableOnce/List/Seq`. It is an easy...
This is a vague idea, but we could make a scalding macro annotation, such as: ```scala @scalding class MyJob(foo: Args) extends Job(args) { } ``` Which could do some standard...
using batched on the map-side dramatically improves the performance of sketch-join.
HadoopMode.newFlowConnector ignores (intentionally) the jobConf because we don't want creepy mutation confusing what the Config is. The problem is, Execution gives no programmatic way to change the Config inside the...
A user has a case class wrapping a `Long`. The tsv had a string that could not be parsed as long, and the use got `0L` for the Long. I...
There is also a mailing list thread: https://groups.google.com/forum/#!topic/scalding-dev/y582r5zmr18 I guess a dependency is missing from the repl. We have a test, but the approach to run it is not just...
Seems like it is creating ambiguous flows since the string name is the same for sink and source even though they write to different directories.
see: #328 and comments there-in. Seems dubious the stated monoid is what is happening, and if it is, we should comment. In any case, we should exercise it in a...