drogue-cloud icon indicating copy to clipboard operation
drogue-cloud copied to clipboard

Figure out why events had been "late" at the hackathon

Open jbtrystram opened this issue 3 years ago • 2 comments

WS does not give an offset when subscribing to kafka

jbtrystram avatar Oct 27 '22 14:10 jbtrystram

I just peeked into the code and tested this a bit. To me it looks like the WebSocket service doesn't to anything special here, and the default is to use an anonymous group, with not offset reset specified, which should result in "latest". So each new connection should stream from the end of the stream. Using drg stream and the example application (websocket-client) confirm this.

So it looks more like a bottleneck somewhere. We had the gateways streaming data to the cloud, the cloud processing messages (around 15 msgs/s), and the consumer receiving events.

I could imagine that there was an issue with the conference WiFi. Leading to a buildup of messages, either from the gateway to the cloud, or from the cloud to the consumer.

ctron avatar Nov 03 '22 11:11 ctron

I also think that a scale test might bring some insight here.

ctron avatar Nov 03 '22 11:11 ctron