osometweet
osometweet copied to clipboard
Improve streaming
According to Twitter's page about consuming streaming data we want to be able to do the below:
- [X] Establish an HTTPS streaming connection to the filter stream endpoint.
- [ ] Asynchronously send POST requests to the filter stream rules endpoint to add and delete rules from the stream.
- [ ] Handle low data volumes – Maintain the streaming connection, detecting Tweet objects and keep-alive signals
- [ ] Handle high data volumes – de-couple stream ingestion from additional processing using asynchronous processes, and ensure client-side buffers are flushed regularly.
- [ ] Manage volume consumption tracking on the client-side.
- [ ] Detect stream disconnections, evaluate and reconnect to the stream automatically.