Improve accuracy by using Twitter

[mzgubic]

Ever approximate a derivative in code with:

h = 0.001 df(x) = (f(x + h) - f(x)) / h

It turns out that this has some pretty serious issues with values that can't be represented in binary. But this simple fix can correct that

temp = x + h h = temp - x

-- https://twitter.com/willkurt/status/1330183861452541953?s=20

@wesselb suggested to insert it here: https://github.com/JuliaDiff/FiniteDifferences.jl/blob/master/src/methods.jl#L270

[mzgubic]

Just adding h = (h + x) - x at the suggested place breaks the high order high adapt test:

Accuracy at high orders, with high adapt: Test Failed at /Users/mzgubic/Projects/FiniteDifferences.jl/test/methods.jl:67
  Expression: ≈((central_fdm(9, 5, adapt = 4, condition = 1))(exp, 1.0), exp(1), atol = 1.0e-7)
   Evaluated: 2.718281705842539 ≈ 2.718281828459045 (atol=1.0e-7)
Stacktrace:
 [1] top-level scope at /Users/mzgubic/Projects/FiniteDifferences.jl/test/methods.jl:67
 [2] top-level scope at /Users/julia/buildbot/worker/package_macos64/build/usr/share/julia/stdlib/v1.5/Test/src/Test.jl:1115
 [3] top-level scope at /Users/mzgubic/Projects/FiniteDifferences.jl/test/methods.jl:67
 [4] top-level scope at /Users/julia/buildbot/worker/package_macos64/build/usr/share/julia/stdlib/v1.5/Test/src/Test.jl:1115
 [5] top-level scope at /Users/mzgubic/Projects/FiniteDifferences.jl/test/methods.jl:5

[wesselb]

Interesting, @mzgubic. This trick to modify the step size does have as a consequence that it makes the step size deviate from the computed "best" step size. Perhaps that's what's now making the adaptation test fail.

I would argue that that adaptation test is really extreme and probably not a setting that would be ever used in practice, so perhaps we could accept the loss of accuracy there if this trick does exhibit good improvement for more commonly occurring use cases.

A better way to integrate this trick is to explicitly account for it in the step size computation, i.e. model an error of |x| eps on h in the denominator of the finite difference. Hmm, I'll think about that.

[Dsantra92]

Is this issue still open?

[mzgubic]

Yep, feel free to tackle