Pankaj Gupta

@NPraneeth good catch.

Thanks Argyris. Sounds exciting. Tbh we don't have any current plans of using beam via Summingbird but this sounds like a great addition to the Summingbird ecosystem. On Thu, Jun...

You're right if we were to simply replace "FlatMap" with "name" in above example for the entire topology then .name is the best choice. That was actually not what I...

Capturing related gitter conversation here: This request boils down to key grouping on left join. We could do this by providing an option on leftJoin indicating that key grouping should...

A problem with keyed leftJoin for online platform is hot keys. Map side aggregation mitigates the hotkey issue for sumByKey operation but doesn't apply here. In case of keyed leftJoin...

The main use-case for this is to enable efficient in-memory caching in the online job. This has to do with the job and not the service. So I think having...

Hot keys would be an issue in some cases and not in others. We should give a choice to enable handling of hot keys. So the option might look like:...

I'm not sure if I understood correctly. Shuffling(distributing equally to all downstream nodes) both hot keys and light keys would mean not doing any grouping at all.

The problem with hot keys is that the number of events can be so high that just handling the network bytes and deserialization can choke the instance. We typically see...

Interesting. So we wrap the key while sending and unwrap on the other side. That would avoid the complexity of adding custom stream grouping but add a little cost in...