VJP Computation (for NonlinearSolve) is extremely inefficient

[avik-pal]

With the latest changes to nonlinear solve, several algorithms compute VJPs and they don't use $J^T v$ rather they use AD to differentiate the internal nonlinear problem, this causes significant increase in solve times.

And if Zygote is not loaded or the function is in place it is even worse as it does FiniteDiff.