Ian O'Connell

Its odd we ran into this and our tests passed, it starts up a local ZK to run storm in local mode for the tests....

Yeah, though we have a storehaus dependency everywhere, and it all depends on util. (Though maybe not finagle i suppose). Just a little concerned about the utility of the tests....

How about having scanByKey instead ? Less Api breakage and sounds like the behavior you want is scan? On Thursday, January 28, 2016, P. Oscar Boykin [email protected] wrote: > Now...

So I don't believe a store works for sumByKey as @jnievelt mentions because in summingbird the store is a dumb thing per say. The bigger impact is the map side...

(Quick scan of internal dashboards for heron showed several jobs using this feature consuming streams at maybe not breakneck levels but one i just looked at was about 10k/sec ingestion...

I don't think you could consume from a summer back then at all, we only supported 1 sumByKey, and those didn't have outputs in the planned graphs afaik.

Not easily if its in use today for that value. Its still a large change to code bases. Though if we wanted to terminate it I do like the idea...

I'm not sure what your suggesting there tbh

Both of those suggestions have massive migration pain @ twitter I would guesstimate. That would be my reservation. If your volunteering for the roll out, Oscar's original suggestion is fine...

I guess the problem is if they do consume it say. Then if the input to the first sumByKey is large there will be no map-side reduce/caching occuring in that...