JSOSolvers.jl icon indicating copy to clipboard operation
JSOSolvers.jl copied to clipboard

R2: model decrease overflow with Float32 on brownden problem

Open d-monnet opened this issue 2 years ago • 1 comments

Hi there, I noticed that R2 overflows with Float32 on the first iteration of Brownden problem from OptimizationProblems.ADNLPProblems. The overflow comes from the way the model decrease is computed: ΔTk = norm_∇fk^2 / σk. Since norm_∇fk is really big, norm_∇fk^2 overflows. I notice that by comparing R2 with my own implementation of R2 that computes the model decrease as ΔTk = dot(∇fk,∇fk / σk) which does not overflow and converges. However I acknowledge that computing ΔTk as in R2 is faster than what I implemented since norm_∇fk is already computed, while my implementation has to compute an extra dot product.

Not sure what you want to do to address that, if it is any problem at all, just thought I'd mention it.

d-monnet avatar May 25 '23 22:05 d-monnet

Maybe something like ΔTk = (norm_∇fk/σk)^2 * σk would help, but underflow might occur if σk is big. I suppose there's no easy fix.

d-monnet avatar May 25 '23 22:05 d-monnet