streamz
streamz copied to clipboard
python-streamz
Real-time stream processing for python
Using `timed_window` in Python 3.7.5 can sometimes lead to a coroutine being awaited more than once. Here's a sample program: ```python async def my_sink(x): print(x) await sleep(2) # raise Exception("Blah!")...
I'm using a pipeline that reads text from file via Apache Tika, performs some pre-processing and writes it into a MongoDB. The following is a truncated version of my script....
Currently we keep a certain buffer length, and emit on every new item. Might be useful to ony emit every Nth item, like Spark-streaming does https://spark.apache.org/docs/2.2.0/streaming-programming-guide.html#window-operations This can be achieved...
The idea of this source would be to simply report back the UUID of each detected BLE 4.0 device at a user defined interval. Interesting streaming use cases can be...
Dask workers' memory shooting up gradually for long running jobs and eventually job crashes when the memory of workers exceeds 80%(or around). Refer to the image below:  Dask is...
Suggest something like the equivalent of dask's `as_completed`, which emits whenever one of the futures contained changes state, or on a regular frequency with the current status of all futures....
Recently I've been trying to split my pipelines up into chunks so the chunks can be re-used as much as possible. One issue I foresee running into is that `scatter`...
One issue I've run into is resetting the state of accumulate nodes. The current operating procedure is if there is a signal to clear the accumulate node, that it should...
I have RabbitMQ and I need to feed the streaming data for parallel execution. So,I opted RabbitMQ-> Streamz -> Dask for this purpose. But I am facing problems for Rabbit-Streamz...
