Sparsity with SparseDiffTools

[matthieugomez]

trafficstars

It would be great to use SparseDifftools in NLSolve, and make it easier for users to work with sparse Jacobians @ChrisRackauckas

[ChrisRackauckas]

The place to start is probably by allowing users to pass a color vector along with a Jacobian prototype, and making use of that.

[matthieugomez]

It could also use automatic sparsity detection right?

[ChrisRackauckas]

It could, be I think we should be a bit careful still before defaulting to it. Cassette is a pretty big dependency, and we don't know everything about the average compile times for using it yet. Standard runtime is around 2 seconds I think, so that might be more of an opt-in thing.

[pkofod]

I'll keep it in mind. A usage example (outside of nlsolve, just a simple application of the tools to a function) would be great!

[matthieugomez]

One syntax could be something like

function OnceDifferentiable(f!, x0, F0, J0; color = matrix_colors(J0), autodiff = true)
   if autodiff == true
      j! = (J, x) -> DiffEqDiffTools.finite_difference_jacobian!(J, f!, x, color = color)
   elseif autodiff == :forward
      jac_cache = ForwardColorJacCache(f!,  x0; color = color)
      j! = (J, x) -> forwarddiff_color_jacobian!(J, f!, x, jac_cache)
   end
   OnceDifferentiable(f!, j!, x0, F0, J0)
end

[ChrisRackauckas]

That matches our plan with DiffEq

[pkofod]

What does J0 need to be here?

[matthieugomez]

It allows the user to indicate the sparsity structure of the Jacobian (which is then used by matrix_colors and the functions that create the Jacobian in place).

That being said, as discussed above, SparseDiffTools makes it possible to get the sparsity structure of f! automatically --- but it still has some drawbacks (e.g. speed). To make J0 optional:

function OnceDifferentiable(f!, x0, F0; J0 = Float64.(sparse(sparsity!(f!,F0, x0))), 
                                            color = matrix_colors(J0), autodiff = true)
 # same function body
end

[pkofod]

To make J0 optional:

I'll figure something out, I was just looking for the basic interface for SparseDiffTools. Thanks!

[ChrisRackauckas]

Here's a blog post that explains it a bit:

http://www.stochasticlifestyle.com/a-collection-of-jacobian-sparsity-acceleration-tools-for-julia/

[pkofod]

Thanks

[ChrisRackauckas]

I think @matthieugomez 's suggestion is all that's needed, and then the differentiation calls in the OnceDifferentiable just need to add the colorvec and sparsity arguments using the colorvec and J0 (it separates sparsity and J because you may want to decompress into a dense matrix, though I assume for NLsolve's purposes you can abstract that away and just make it the same for both).

[pkofod]

I think @matthieugomez 's suggestion is all that's needed, and then the differentiation calls in the OnceDifferentiable just need to add the colorvec and sparsity arguments using the colorvec and J0 (it separates sparsity and J because you may want to decompress into a dense matrix, though I assume for NLsolve's purposes you can abstract that away and just make it the same for both).

Agreed, though Matthieu's code doesn't actually make it optional, it forces it to be the default, but I will make a version where you can pass the required info.

[ChrisRackauckas]

Now sparsity with FiniteDiff.jl for numerical and SparseDiffTools for the AD part. Check the implementation in OrdinaryDiffEq for a way to use both to do something similar to what NLsolve's current interface does.

[pkofod]

I will switch over in NLSolversBase then it should be reflected here "automatically" (we may need to introduce some keywords for ease of use)