more support for out-of-place trust-region solvers

[longemen3000]

this is focused in the NWI trust region, but TCG also supports out of place now (there was some work on NTR, but some parts are still missing)

summary of the changes:

• update_H!(H,h,lamda) -> update_H!(mstyle, H,h,lamda)
• new function: trs_supports_outofplace(trs), that turns the support for out-of-place solvers for an specific trust region method.
• dot(x, H*x) -> dot(x, H, x) (available since julia 1.4, it should reduce an allocation in the inplace methods)

[codecov[bot]]

Codecov Report

Attention: 20 lines in your changes are missing coverage. Please review.

Comparison is base (6fd621a) 76.91% compared to head (97a2dc7) 77.11%.

Files	Patch %	Lines
src/globalization/trs_solvers/solvers/NTR.jl	61.29%	12 Missing :warning:
src/globalization/trs_solvers/solvers/NWI.jl	79.16%	5 Missing :warning:
src/globalization/trs_solvers/TRS.jl	0.00%	1 Missing :warning:
src/globalization/trs_solvers/root.jl	96.29%	1 Missing :warning:
src/globalization/trs_solvers/solvers/TCG.jl	96.42%	1 Missing :warning:

Additional details and impacted files

@@            Coverage Diff             @@
##           master      #66      +/-   ##
==========================================
+ Coverage   76.91%   77.11%   +0.20%     
==========================================
  Files          54       54              
  Lines        2807     2867      +60     
==========================================
+ Hits         2159     2211      +52     
- Misses        648      656       +8     

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.

[longemen3000]

The tests here pass with rosenbrock and NWI, but fails with my test case:

import Clapeyron, ForwardDiff
const C = Clapeyron

#obtain critical point of water with PC-SAFT eos
function test_critical_point()
  model = C.PCSAFT("water")
  function f_crit_static(Fx, x)
    Ts = T_scale(model,SVector(1.0))
    T_c = x[1]*Ts
    V_c = exp10(x[2])
    ∂²A∂V², ∂³A∂V³ = ∂²³f(model, V_c, T_c, SA[1.0])
    F1 = -∂²A∂V²
    F2 = -∂³A∂V³
    return SVector(F1,F2)
  end
  f_crit_static(x) = f_crit_static(nothing, x)
  j_crit_static(J,x) = ForwardDiff.jacobian(f_crit_static,x)
  fj_crit_static(F,J,x) = f_crit_static(x),j_crit_static(J,x)
  obj = NLSolvers.VectorObjective(
          f_crit_static,
          j_crit_static,
          fj_crit_static,
          nothing,
  )
  prob_static =  NLSolvers.NEqProblem(obj; inplace=false)
  x01,x02 = C.x0_crit_pure(model)
  x0_static= SVector(x01,x02)
  NLSolvers.solve(prob_static, x0_static, TrustRegion(Newton(), NWI()), NEqOptions(maxiter = 20))
end

on the allocating version: this is the output of the trust region solver:

spr = (p = [-0.119191334506046, 0.04355412894426012], mz = -4.018965005920141e36, interior = true, λ = 0.0, hard_case = false, solved = true, Δ = 20.0)
spr = (p = [-0.0027029758072131525, 0.04499612541909675], mz = -4.7551669760870206e35, interior = true, λ = 0.0, hard_case = false, solved = true, Δ = 35.0)
spr = (p = [0.018511262372664327, 0.045777571320208765], mz = -6.205904430656341e34, interior = true, λ = 0.0, hard_case = false, solved = true, Δ = 61.25)
spr = (p = [0.013106086890848268, 0.04429461125778653], mz = -8.344042103677409e33, interior = true, λ = 0.0, hard_case = false, solved = true, Δ = 107.1875)
spr = (p = [0.002156155676237826, 0.03809921614778], mz = -1.063615592722335e33, interior = true, λ = 0.0, hard_case = false, solved = true, Δ = 187.578125)
spr = (p = [-0.004636514979904277, 0.025680354545900588], mz = -1.1154169725112833e32, interior = true, λ = 0.0, hard_case = false, solved = true, Δ = 328.26171875)
spr = (p = [-0.0037736725583891557, 0.010414501514063087], mz = -6.834517914423543e30, interior = true, λ = 0.0, hard_case = false, solved = true, Δ = 574.4580078125)
spr = (p = [-0.0007013620414804606, 0.0014772805678870081], mz = -9.525776840814146e28, interior = true, λ = 0.0, hard_case = false, solved = true, Δ = 1005.301513671875)
spr = (p = [-1.4438428337513423e-5, 2.7098407728549498e-5], mz = -3.2082145157529907e25, interior = true, λ = 0.0, hard_case = false, solved = true, Δ = 1759.2776489257812)
spr = (p = [-5.073980186604831e-9, 9.04901950919703e-9], mz = -3.700814555101485e18, interior = true, λ = 0.0, hard_case = false, solved = true, Δ = 3078.735885620117)
spr = (p = [-7.883218328763044e-15, -7.778461704842644e-15], mz = -577.1355732863631, interior = true, λ = 0.0, hard_case = false, solved = true, Δ = 5387.787799835205)
spr = (p = [3.845471800641081e-16, 4.870267452997498e-16], mz = -256.50452171776385, interior = true, λ = 0.0, hard_case = false, solved = true, Δ = 9428.628649711609)
spr = (p = [3.607523191190752e-17, 1.5088257208636439e-16], mz = -256.5045209330882, interior = false, λ = 2.4796342977323244e25, hard_case = false, solved = false, Δ = 1.5513534097936166e-16)

whereas the out-of-place version returns:

spr = (p = [-0.119191334506046, 0.043554128944260126], mz = -4.018965005920142e36, interior = true, λ = 0.0, hard_case = false, solved = true, Δ = 20.0)
spr = (p = [-0.0027003500478230986, 0.04499632231262638], mz = -4.755166976087023e35, interior = true, λ = 0.0, hard_case = false, solved = true, Δ = 35.0)
spr = (p = [0.018510197652017554, 0.04577760432131138], mz = -6.205942098689902e34, interior = true, λ = 0.0, hard_case = false, solved = true, Δ = 61.25)
spr = (p = [0.013104649946379613, 0.04429447425986432], mz = -8.344077069894073e33, interior = true, λ = 0.0, hard_case = false, solved = true, Δ = 107.1875)
spr = (p = [0.0021561276055823263, 0.03809921508070954], mz = -1.0636125973524221e33, interior = true, λ = 0.0, hard_case = false, solved = true, Δ = 187.578125)
spr = (p = [-0.004636552983922482, 0.025680336293903983], mz = -1.1154135772229733e32, interior = true, λ = 0.0, hard_case = false, solved = true, Δ = 328.26171875)
spr = (p = [-0.0037737304612361034, 0.010414445181253269], mz = -6.834487533690134e30, interior = true, λ = 0.0, hard_case = false, solved = true, Δ = 574.4580078125)
spr = (p = [-0.0007013635448527738, 0.0014772628407236513], mz = -9.525623669547625e28, interior = true, λ = 0.0, hard_case = false, solved = true, Δ = 1005.301513671875)
spr = (p = [-1.443704729940474e-5, 2.7098884597450056e-5], mz = -3.2080808621100005e25, interior = true, λ = 0.0, hard_case = false, solved = true, Δ = 1759.2776489257812)
spr = (p = [-5.068404359382654e-9, 9.05447388244233e-9], mz = -3.700854793463538e18, interior = true, λ = 0.0, hard_case = false, solved = true, Δ = 3078.735885620117)
#difference on mz
spr = (p = [-1.8265991947435736e-15, 7.918581472135602e-16], mz = -124148.18807046987, interior = true, λ = 0.0, hard_case = false, solved = true, Δ = 5387.787799835205)
spr = (p = [-6.729569458391707e-16, -3.0869047683300955e-15], mz = -113118.4936590613, interior = true, λ = 0.0, hard_case = false, solved = true, Δ = 2693.8938999176025)
spr = (p = [-1.3459155687153562e-15, 3.2686829703639614e-16], mz = -50274.88608109882, interior = true, λ = 0.0, hard_case = false, solved = true, Δ = 4714.314324855804)
spr = (p = [2.88410656444924e-16, 3.9429121117859366e-16], mz = -256.50452129970915, interior = true, λ = 0.0, hard_case = false, solved = true, Δ = 8250.050068497658)
spr = (p = [2.8841587101657233e-16, 3.942962412761004e-16], mz = -256.50452129973655, interior = true, λ = 0.0, hard_case = false, solved = true, Δ = 1.2354744340255133)
spr = (p = [2.8841587101657233e-16, 3.942962412761004e-16], mz = -256.50452129973655, interior = true, λ = 0.0, hard_case = false, solved = true, Δ = 1.2354744340255133)
spr = (p = [2.8841587101657233e-16, 3.942962412761004e-16], mz = -256.50452129973655, interior = true, λ = 0.0, hard_case = false, solved = true, Δ = 1.2354744340255133)
spr = (p = [2.8841587101657233e-16, 3.942962412761004e-16], mz = -256.50452129973655, interior = true, λ = 0.0, hard_case = false, solved = true, Δ = 1.2354744340255133)
spr = (p = [2.8841587101657233e-16, 3.942962412761004e-16], mz = -256.50452129973655, interior = true, λ = 0.0, hard_case = false, solved = true, Δ = 1.2354744340255133)
spr = (p = [2.8841587101657233e-16, 3.942962412761004e-16], mz = -256.50452129973655, interior = true, λ = 0.0, hard_case = false, solved = true, Δ = 1.2354744340255133)
spr = (p = [2.8841587101657233e-16, 3.942962412761004e-16], mz = -256.50452129973655, interior = true, λ = 0.0, hard_case = false, solved = true, Δ = 1.2354744340255133) 

[pkofod]

I'll look into it, thanks