Should Provide Option for Graceful Handling of non-Finite Floats

[Wardrop]

Currently, the JSON library bombs out with an exception whenever it hits a non-finite float such as NaN or Infinite. It does provide an option to ignore these values as to include them in the JSON output as-is, but that generates invalid JSON which only moves the problem to the client. I'd like to suggest a new option be provided to allow one more control over how these non-finite (or otherwise invalid) values should be handled.

The option could be called something like "replace_nan" or something like that to keep it consistant with the "allow_nan" option. The possible values could be true, false or a proc. If false, the behaviour remains as it is currently. If true, the library replaces non-finite values with a relatively safe default, e.g. nil or 0. If a proc is given, the library should call the proc whenever a non-finite value is encountered, passing it the value itself. The return value of the proc is then used as the replacement value.

I don't think I need to justify this request with a use-case, but to give you an example in case you want one, I've just been called into work from holidays to fix an issue with an application I've written. It uses the EXIFR library to parse EXIF data from JPEG's and TIFF's. This typically generates NaN values which are serialised to the database. When this data is requested by the client, I need to perform a to_json on the serialised object which may include NaN's. Currently, it bombs out, and short of monkey-patching the #to_json method on the Float class, there's nothing I can do to work around this, except for some kind of elaborate recursive traversal of the object before it's jsoned, which is obviously a complex, fragile and slow solution.

[Wardrop]

Turns out monkey patching C code is rather impossible in this case, because many methods (such as Array#to_json and Object#to_json) call internal C methods directly. It's not like they all delegate to a method exposed in ruby, which would have been a more convenient implementation for monkey patchers.

That makes this feature addition even more urgent. Anyone know how to force this library to use the pure ruby generator, as I can at least monkey patch that.

[Wardrop]

Been spending the last few hours trying to find a work-around, and I've got nothing. Found out how to force which generator to use, but as soon as the default 'json' library from stdlib is included (which any 3rd-party library may itself include), then it's almost impossible to keep JSON::Ext::Generator out of the picture.

I've had to disable EXIF data in my application until a work-around to this is provided. Makes me wonder how anyone else manages to work around this? Absolutely ready to pull my hair out.

[flori]

If you want to use the pure generator you can require 'json/pure'. There is also a variant of this gem (json_pure) that comes without a c extension, so require 'json' will always be the pure parser/generator.

However, my first attempt at implementing your replace_nan idea can be seen here:

https://github.com/flori/json/tree/replace-nan

It already works in the pure and c-version by replacing infinites with null, but there is more work still to be done. Maybe you can point your bundler :ref at an appropriate commit in this branch if you cannot wait that long.

[Wardrop]

Thanks for the response Flori. I've actually tried the json_pure gem. The problem is, the C generator is still loaded and set, which I guess makes sense given that it's loading the stdlib version. I can't manually rely on requiring "json/pure" explicitly as there are other libraries (including 3rd party) that do a require "json" which loads the C generator from stdlib, which obviously amends all the #to_json methods among other things - so even manually setting the JSON.generator to JSON::Pure::Generator doesn't work as the #to_json methods are mismatched and fail to work (at least when monkey-patched).

I might do what you said and reference a commit from my Gemfile for the time being. I'll see if that works.

Thanks Flori.

[Wardrop]

I've just been playing with that branch that has replace_nan. Indeed, it works, however I'm looking for a way to set replace_nan to true by default, so if I do a [0/0.0].to_json, it produces "[null]". The JSON module has a #dump_default_options method, but that doesn't affect the defaults of #generate (which is what #to_json is as far as I can tell). Do you know a way I can set defaults for generate?

I found the JSON::SAFE_STATE_PROTOTYPE constant which refers to a State object. Should I modify this object to set defaults, or is there a better way?