Possible Ruby 1.9 encoding issue

[andruby]

BERT using Ruby 1.9 can encode the integer 2282389, but when it is sent to an Ernie server, the call hangs. I don't know yet if this is an issue with bertrpc (ruby client) or with Ernie (the server).

https://gist.github.com/1012748

BERTRPC::Service.new("localhost", 9999).call.ernie.echo(2282389)

[andruby]

Using ruby-debug I have found that the line where it hangs is

ruby-1.9.2-p180/gems/bertrpc-1.3.0/lib/bertrpc/action.rb:29

This is the code on that line:

r, w, e = IO.select([sock], [], [], timeout)

So I guess the IO.select waits indefinitely for the socket to be ready.

In the Ernie access log there is no line for the request so I am not sure where the "hang" really occurs. @mojombo: do you think this is an encoding issue?

[applio]

I am encountering the same issue, I believe. I first encountered it with passing the integer 1229952 to a function served/exposed by the Ernie server. I can reproduce the same issue with the magic number 2282389.

The error seen at the erlang shell suggests that the ruby/bertrpc call communicates its payload to ernie but an error occurs in the erlang code that results in ernie not responding to the waiting IO.select on the ruby end. Observed error: =ERROR REPORT==== 1-Jul-2011::13:34:39 === Error in process <0.210.0> with exit value: {badarg,[{erlang,binary_to_term,[<<57 bytes>>]},{ernie_server,receive_term,2}]}

I am invoking my integer-expecting function via ruby-1.9.2-p180 and ernie is running on erlang R13B04. I have gems for bert (1.1.2), bertrpc (1.3.0), and ernie (2.5.2) installed on the ruby side.

Exploring a bit, it seems that there are a range of problematic integers around the examples mentioned at the start of this post. One such range includes the 2282389 magic number: 2282368 to 2282431 (inclusive) all provoke the described issue. Another range includes my 1229952 magic number: 1229952 to 1230015 (inclusive) all provoke the same issue. Interestingly, both ranges include 64 numbers.

[applio]

Turning on logging in ernie helps show that the input payload received by the ernie server was indeed an odd number of bytes (57 bytes in my case), which provokes the binary_to_term/1 error. This suggests that the problem originates in the ruby-side code that generated the payload. Investigating that further, the creation of the payload comes directly from the bert module. Within the bert module's encode.rb, the function write_fixnum() generates the part of the payload for the integer but the static function self.encode() that called it returns a StringIO.string. Looking at the issues on the bert repo, it sounds like lucaspiller has identified the underlying issue (only happens when using ruby 1.9.x and/or unfortunate choice of default encoding) and already fixed it. His pull request for his fix dates back to October 2010.

Manual testing of lucaspiller's patch -- this is what bert 1.1.2 causes bertrpc 1.3.0 to unfortunately do when using ruby 1.9.x:

irb(main):025:0> BERT::Encode.encode(2282389)
=> "\x83b\u0000\"ӕ"

This is what lucaspiller's patch essentially changes in bert, resulting in a proper payload:

irb(main):026:0> BERT::Encode.encode(2282389).force_encoding("ASCII-8BIT")
=> "\x83b\x00\"\xD3\x95"

[runemadsen]

I think I'm running into a very similar problem here. @applio can you describe what "unfortunate choice of default encoding" means? What encoding should I choose to make this problem go away?

[applio]

@runemadsen , the default encoding in ruby 1.9 differs from ruby 1.8 which triggers the problem when using 1.9 and not 1.8. My generalization to "unfortunate choice of default encoding" was meant to include the notion that you could reproduce the issue using ruby 1.8 if you changed the encoding in the appropriate places -- I have not tried this myself but it seems probable. There is not an easy fix along the lines of just changing the encoding at some high/low level to solve all ills here -- @lucaspiller has, I believe, correctly identified what's needed.

I would recommend patching your copy of bert, using the patch from @lucaspiller . I've done that in my own fork at https://github.com/applio/bert which might include a few other helpful patches, if it helps.

[runemadsen]

Awesome. Thanks so much. Actually, the current HEAD of the bert repo seems to have fixed this problem also: https://github.com/mojombo/bert/blob/master/lib/bert/encode.rb. It disappeared when I installed via bundler :git => "git://github.com/mojombo/bert.git"

[applio]

Cool -- even better.