Vendor Base.Ryu

[nhz2]

From what I can tell, Base.Ryu is internal, so it may be removed in a future Julia version.

This PR vendors that code and other internal functions from Base that Ryu uses.

[codecov[bot]]

Codecov Report

:x: Patch coverage is 90.12075% with 90 lines in your changes missing coverage. Please review. :white_check_mark: Project coverage is 90.09%. Comparing base (3f7d15f) to head (69109d4).

Files with missing lines	Patch %	Lines
src/ryu/shortest.jl	86.54%	46 Missing :warning:
src/ryu/fixed.jl	87.91%	18 Missing :warning:
src/ryu/exp.jl	91.00%	17 Missing :warning:
src/ryu/Ryu.jl	91.66%	6 Missing :warning:
src/write.jl	50.00%	3 Missing :warning:

Additional details and impacted files

@@            Coverage Diff             @@
##           master     #384      +/-   ##
==========================================
+ Coverage   89.89%   90.09%   +0.20%     
==========================================
  Files           7       12       +5     
  Lines        1316     2221     +905     
==========================================
+ Hits         1183     2001     +818     
- Misses        133      220      +87     

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.

:rocket: New features to boost your workflow:

• :snowflake: Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[quinnj]

I don't think this is a good idea to vendor:

• The Base Ryu code is still evolving/maturing; there have been at least 6 PRs this year massaging the code
• Vendoring means now we have a much higher code maintenance burden to try and keep synced w/ Base
• I think we can do something much simpler by doing a compat check on the presence of Base.Ryu and do standard printf calls otherwise.

I'm more worried about the long-term maintenance of this vendored code than Base breaking or slight performance hit of doing printf via compat.

[nhz2]

The point of vendoring this code is that Base.Ryu is free to evolve without breaking JSON.jl

There is no need for this to be kept in sync with Base because it is its own self-contained separate copy.

A check on the existence of Base.Ryu doesn't help, because the functions may still exist, but what they do may change.

JSON.jl is also used to develop Julia itself, so JSON.jl cannot react to changes in Base internals like a normal package.

[quinnj]

The point of vendoring this code is that Base.Ryu is free to evolve without breaking JSON.jl

Base is already free to evolve? I know package eval is run to check breakage on things, but for something internal like Ryu, I don't think it would stop changes from happening.

There is no need for this to be kept in sync with Base because it is its own self-contained separate copy.

That sounds problematic to me; bug fixes or even worse, security issues, would then not flow through to JSON eventually via upgrades.

A check on the existence of Base.Ryu doesn't help, because the functions may still exist, but what they do may change.

It seems practical to add sophisticated checks via reflection on the exact methods + shape expected and use if present, and if not exact matching, fall back to non-vendored. It's notable since I wrote the original Ryu implementation, that the main interfaces for the used functions hasn't changed (> 6 years ago).

JSON.jl is also used to develop Julia itself, so JSON.jl cannot react to changes in Base internals like a normal package.

That doesn't seem particularly fair or relevant here; if Base is using JSON.jl to develop Julia, then they need to be ready to account for changes/breakage in the package like if they relied on any other package. As far as I'm aware, there's no formal documentation or agreement on how this works or is expected to work, so, in my mind, doesn't exist. If JSON.jl is actually critical in a key way to Base, then it should be promoted to a proper stdlib, which I believe is the more formal way of saying "this is functionality we want to be tied/tracked/evaluated against Base more closely than other packages".

I've opened https://github.com/JuliaLang/julia/issues/59763 to further discuss formally exposing an API to the Ryu internal functions and I think that will provide a practical answer here.

[hhaensel]

I think, @quinnj 's view is right here. JSON is a package maintained by Julia, if Base.Ryu happens to break JSON in the future it can be fixed at that time.