Christopher J. 'CJ' Wright

Results 173 comments of Christopher J. 'CJ' Wright

You were the one who brought it up, https://github.com/mrocklin/streamz/pull/50#issuecomment-322848922

I don't know if we can filter out futures without interrogating them (and thus needing to be greedy). I was thinking about a stream which contained null/no-op values.

We could call it something else, I'm not too attached to the name. I'm mostly interested in functionality similar to what we have in the core streamz module, where values...

Maybe, although it seems like an undue burden on the user code to handle `DaskStream` specific filter logic. Maybe there is a middle layer between the two.

One could imagine a version of `buffer` which emits data as soon as the future is finished. This would break the ordering but could be ok for some applications.

Is there some code which says that the source died? If so then you could maybe push all the data into a buffer while waiting for the source to come...

I've used a combination of joining nodes, filter, and pluck to essentially gate certain paths. Maybe two gates would work for pushing data directly into the processing or into a...

Attn: @sbillinge

Great (to the need it as well)! Another option is to include the metadata with the data and scrape the metadata off before doing the transformation and then putting it...

Hmm I didn't think of 2. I was thinking of having metadata-data-metadata-... and then having the streams nodes (potentially including the base class) behave differently for data and metadata.