Silent fail / exit with streamz >= 0.4.0 when pipeline is being run in a daemon process

[michigraber]

I've been using streamz to consume events from kafka. For that I built a pipeline which I run in a daemon process, managed by the python daemon package (https://pypi.org/project/python-daemon/). The pipeline ultimately writes to some elasticsearch instance.

This worked great with streamz version 0.3.0 with which I started developing. Now I wanted to update to v0.5.0 and discovered that while I can still perfectly run the streamz pipeline in the current python process, the daemon running the pipeline suddenly stops without any further detectable error in the pipeline itself after consuming the first message. I downgraded to v0.4.0, then v0.3.0 and observed that the fail is introduced by the 0.3.0 -> 0.4.0 step.

Does anybody has an idea what introduced the problem at hand? Is there an easy way to fix the issue with the later versions?

.. I somehow suspect that it has something to do with the tornado processes started ..?

[amwi04]

Having same problem with kafka but it seams to work in jupyter nootbook

[martindurant]

@michigraber , sorry you didn't get a response to this. Would you mind trying with the latest kafka and with #233 ?