datastream.io icon indicating copy to clipboard operation
datastream.io copied to clipboard

streamz integration

Open canagnos opened this issue 7 years ago • 3 comments

canagnos avatar Feb 20 '18 06:02 canagnos

I assume you're referring to this: https://streamz.readthedocs.io/

What I'm wanting to do is have a WebSocket in Python that connects to a server, and then the server continuously sends metrics to dsio, and dsio sends back any anomalies it finds via the WebSocket.

erichocean avatar Feb 22 '18 01:02 erichocean

Yes, that's the project we are referring to. With regards to web sockets would you connect directly, or via Kafka? Do you have a reason to not want a queue in your stack? We're interested!

canagnos avatar Feb 28 '18 15:02 canagnos

Yeah, I would be connecting directly. We don't want/need the overhead of managing Kafka just for message queuing, we already have an orchestrator node that maintains information about the behavior of our system, and that we want to connect to the dsio library to learn the behavior of our system by feeding it data points, and then notify our orchestrator re: outliers so we can take the appropriate action(s).

erichocean avatar Mar 01 '18 00:03 erichocean