pyjson5 icon indicating copy to clipboard operation
pyjson5 copied to clipboard

Switch from fast_double_parser to fast_float

Open sirosen opened this issue 9 months ago • 2 comments

First off, thanks for maintaining this library! I'm trying to pitch in a bit to help and make sure things stay healthy. It's my first time working with Cython, so I'll be a little slow to make changes.


This PR swaps the fast_double_parser for fast_float, per the deprecation warning cited in #83. It all works cleanly except for a small number of tests regarding scientific notation.

I could use some input on how to push this over the finish line. None of the fast_float behaviors matches what the JSON5 tests expect, and the one which is labeled json_or_infnan seems like the best fit for the spec in spirit.

Right now, I'm seeing pyjson5.loads(1e2.3) return 100, which is definitely wrong. But I haven't yet figured out where the issue is. I would guess (from the above) that somehow it's truncating the data to 1e2.

sirosen avatar May 19 '25 05:05 sirosen

Thank you very much for working on that! I'll try to have a look in the next few days.

(I wonder why the CI did not run for you PR, I'll have to investigate that.)

Kijewski avatar May 19 '25 06:05 Kijewski

I was able to confirm that the string "1e2.3", when passed to fast_float, parses "successfully" as 100.0. I couldn't find a way to get it working using clever fast_float usage, but once I satisfied myself that it wasn't possible, I added a manual little function to guard against weird invalid exponents (any non-digit char after the sign).

I think it's now in good shape -- all tests pass -- and I hopefully have some more contributions to follow!

sirosen avatar May 19 '25 22:05 sirosen