osometweet icon indicating copy to clipboard operation
osometweet copied to clipboard

Improve streaming

Open mr-devs opened this issue 2 years ago • 2 comments

According to Twitter's page about consuming streaming data we want to be able to do the below:

  • [X] Establish an HTTPS streaming connection to the filter stream endpoint.
  • [ ] Asynchronously send POST requests to the filter stream rules endpoint to add and delete rules from the stream.
  • [ ] Handle low data volumes – Maintain the streaming connection, detecting Tweet objects and keep-alive signals
  • [ ] Handle high data volumes – de-couple stream ingestion from additional processing using asynchronous processes, and ensure client-side buffers are flushed regularly.
  • [ ] Manage volume consumption tracking on the client-side.
  • [ ] Detect stream disconnections, evaluate and reconnect to the stream automatically.

mr-devs avatar Sep 07 '21 18:09 mr-devs