Generator does not actually set encoding

[headius]

Here's the reduced test case I came up with, from a larger case by @rkh:

# encoding: utf-8
require 'json'
x = 'foó'
p x.to_json
p x.to_json.encoding

The resulting json string ends up encoded as ASCII-8BIT. My probably-wrong patch is:

diff --git a/java/src/json/ext/Generator.java b/java/src/json/ext/Generator.java
index ecceb27..6ea5aa3 100644
--- a/java/src/json/ext/Generator.java
+++ b/java/src/json/ext/Generator.java
@@ -391,6 +391,7 @@ public final class Generator {
                 }

                 session.getStringEncoder().encode(src.getByteList(), buffer);
+                buffer.setEncoding(src.getByteList().getEncoding());
             }
         };

I doubt this is right, but I could not find anywhere that json lib is actually setting the final encoding of the target.

[flori]

I cannot reproduce this problem:

jruby-1.6.7.2 :090 > require 'json'
 => true
jruby-1.6.7.2 :010 > JSON.generator
 => JSON::Ext::Generator 
jruby-1.6.7.2 :011 > JSON::VERSION
 => "1.7.3" 
jruby-1.6.7.2 :012 > RUBY_DESCRIPTION
 => "jruby 1.6.7.2 (ruby-1.9.2-p312) (2012-05-01 26e08ba) (Java HotSpot(TM) 64-Bit Server VM 1.6.0_31) [darwin-x86_64-java]" 
jruby-1.6.7.2 :013 > x = 'foó'
 => "foó" 
jruby-1.6.7.2 :014 > x.encoding
 => #<Encoding:UTF-8> 
jruby-1.6.7.2 :015 > x.to_json
 => "\"foó\"" 
jruby-1.6.7.2 :016 > x.to_json.encoding
 => #<Encoding:UTF-8>

The encoding is set in the method

private static abstract class Handler<T extends IRubyObject>

before the new string is returned.

[rkh]

git clone gh
cd gh
bundle install
bundle exec rspec ./spec/response_spec.rb:5 

[rkh]

Or, using your example:

>> x = 'foó'
=> "foó" 
>> x.encoding
=> #<Encoding:UTF-8> 
>> JSON.load(x.to_json).encoding
=> #<Encoding:ASCII-8BIT>

[flori]

Ah, that's interesting. It could be the load, does parse work instead?

[rkh]

Nope, same result.

[flori]

Ok, I am trying to reproduce this again when I am at a real Computer.

[flori]

I tried "my" example from above and it works here:

(flori@silverblade:gh [master] ☯@gh ->0)$ jruby --1.9 -S irb
jruby-1.6.7.2 :001 > require 'json'
 => true 
jruby-1.6.7.2 :002 > x = 'foó'
 => "foó" 
jruby-1.6.7.2 :003 > x.encoding
 => #<Encoding:UTF-8> 
jruby-1.6.7.2 :004 > JSON.load(x.to_json).encoding
 => #<Encoding:UTF-8> 
jruby-1.6.7.2 :005 > JSON::VERSION
 => "1.7.3" 
jruby-1.6.7.2 :006 > RUBY_DESCRIPTION
 => "jruby 1.6.7.2 (ruby-1.9.2-p312) (2012-05-01 26e08ba) (Java HotSpot(TM) 64-Bit Server VM 1.6.0_31) [darwin-x86_64-java]" 

Are we using the same JSON and jruby versions?

[flori]

So, when I run your spec an exception is raised in JRuby, isn't this rather #138 than #137?