httpstream icon indicating copy to clipboard operation
httpstream copied to clipboard
verified publisher icon araddon

Using delimited to increase performance?

Open lateefj opened this issue 12 years ago • 2 comments

Wasn't sure if using delimited would improve performance but it would allow for just reading the exact number of bytes. Docs: https://dev.twitter.com/docs/streaming-apis/parameters#delimited. I noticed that Python implementation https://github.com/tweepy/tweepy/blob/master/tweepy/streaming.py uses and so I looked into it. Thoughts?

lateefj avatar Jul 14 '13 18:07 lateefj

This is probably not a bad idea, currently it scans for new lines https://github.com/araddon/httpstream/blob/master/stream.go#L174 , doubt the impact would be noticeable unless doing many more than currently max available from twitter stream (3050/min). The underlying connections, buffering, re-connections all probably could be refactored and cleaned up. So, ill leave it as an open issue for now.

araddon avatar Jul 14 '13 21:07 araddon

True, I noticed the tweepy implementation to be streaming a lot faster but I would think this is because of some other issue not the Reader.

lateefj avatar Jul 14 '13 23:07 lateefj