ForwardDiff.jl icon indicating copy to clipboard operation
ForwardDiff.jl copied to clipboard

Resolve `NaN` in hypot near zero

Open timholy opened this issue 2 years ago • 1 comments

This checks whether hypot of the value component is zero, and if so switches to a next-order method.

~This may be slightly controversial since it exploits the difference between 0.0 and -0.0 to give the correct sign behavior at the origin.~ This exploits the difference between 0.0 and -0.0 to provide the same behavior exhibited for abs:

julia> ForwardDiff.derivative(abs, 0.0)
1.0

julia> ForwardDiff.derivative(abs, -0.0)
-1.0

# and now:
julia> f(x) = hypot(x, 0, 0)
f (generic function with 1 method)

julia> ForwardDiff.derivative(f, 0.0)
1.0

julia> ForwardDiff.derivative(f, -0.0)
-1.0

This implementation is consistent with the limit -> 0, e.g.,

julia> hypotvec(v) = hypot(v...)
hypotvec (generic function with 1 method)

julia> ForwardDiff.gradient(hypotvec, [0.0, -0.0, 0.0])
3-element Vector{Float64}:
  0.5773502691896258
 -0.5773502691896258
  0.5773502691896258

# for comparison:
julia> ForwardDiff.gradient(hypotvec, [1e-12, -1e-12, 1e-12])
3-element Vector{Float64}:
  0.5773502691896257
 -0.5773502691896257
  0.5773502691896257

Note this only covers the 3-arg version. For the 2-arg version I think a similar fix must be made in DiffRules.

timholy avatar Oct 13 '23 12:10 timholy

Codecov Report

All modified lines are covered by tests :white_check_mark:

Comparison is base (d300209) 89.65% compared to head (482dde0) 87.41%.

Additional details and impacted files
@@            Coverage Diff             @@
##           master     #669      +/-   ##
==========================================
- Coverage   89.65%   87.41%   -2.25%     
==========================================
  Files          11       10       -1     
  Lines         967      898      -69     
==========================================
- Hits          867      785      -82     
- Misses        100      113      +13     
Files Coverage Δ
src/dual.jl 80.73% <100.00%> (-1.42%) :arrow_down:

... and 9 files with indirect coverage changes

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.

codecov[bot] avatar Oct 13 '23 13:10 codecov[bot]