twitter-stream
twitter-stream copied to clipboard
HTTP::Parser::Error: Could not parse data entirely (EDIT: with a fix)
Tested against master today - when I do this to change what I track:
stream.options[:content] = new_content
stream.immediate_reconnect
I get this error (after 2 on_reconnect calls):
[GEM_ROOT]/gems/twitter-stream-0.1.14/lib/twitter/json_stream.rb:121:in `<<'
[GEM_ROOT]/gems/twitter-stream-0.1.14/lib/twitter/json_stream.rb:121:in `receive_data'
[GEM_ROOT]/gems/eventmachine-1.0.0.beta.4/lib/eventmachine.rb:179:in `run_machine'
[GEM_ROOT]/gems/eventmachine-1.0.0.beta.4/lib/eventmachine.rb:179:in `run'
I updated to master and it still applies.
Does anyone else meet this issue?
Just realized that the first line in the code I maintain was doing a bit of monkey patching:
module Twitter
class JSONStream < EventMachine::Connection
attr_reader :options
end
end
My guess is that the use case may not be currently supported by the gem, then. I'll dive more and see if I can make this work somehow.
Any hint welcome if you read this!
Figured out: on reconnect, the @parser
also needs to be reset, otherwise it will attempt to continue parsing as defined by the previous connection.
I made a fix that works for me, although beware, I'm only discovering this gem so there will be a better way IMO.
https://github.com/thbar/twitter-stream/compare/fix-for-27-could-not-parse-data-entirely
How to use:
Signal.trap('USR1') do
@should_restart = true
end
EventMachine::add_periodic_timer(1) do
if @should_restart
tracked = get_updated_keywords.map { |e| CGI.escape(e) }.join(',')
stream.reconnect_with_updated_config do |options|
options[:content] = "track=#{tracked}"
end
@should_restart = false
end
end
Let me know if you find a better way to do this.
@mgreen @rud @abuiles if this stuff makes sense, let me know and I'll wrap a proper pull-request! (I suspect there's probably a built-in way to achieve that already, but unsure).
So how does one use this again? Just put in that code anywhere and it runs?
n'thing tibbon: can you provide some examples of a configuration or implementation of this fix or is this something that is going to be included in a release of the gem?
@tibbon see my sample above; basically you need a way to detect you need to change the keywords (for my customer's app here I used a signal because it was convenient to inform the program from outside), then use reconnect_with_updated_config
. (all this after applying my patch at https://github.com/thbar/twitter-stream/compare/fix-for-27-could-not-parse-data-entirely).
Hope this helps!