SInDy example with Neural ODE

[aelmokadem]

Hi,

I was hoping someone could point me to an example that integrates Neural ODEs and SInDy together on just the unknown flux in an ODE. The Neural ODE example could be something like:

ann = FastChain(FastDense(2, 32, tanh),FastDense(32, 32, tanh), FastDense(32, 1))
p = initial_params(ann)
function NeoODE(du, u, p, t)
    z = ann(u, p)
    du[1] = -z[1]
    du[2] = z[1] - 10*u[2]
end

You can see that we only want to optimize the z flux here with a neural network. How would we adapt the code to run SInDy just on that flux?

Perhaps a naive side question: Does the sign in front of the neural network matter (be it + or - to match the direction of the flux)?

Thanks, Ahmed.

[ChrisRackauckas]

The code from the Universal Differential Equations paper is here: https://github.com/ChrisRackauckas/universal_differential_equations. Specifically, this is the example that first demonstrated it:

https://github.com/ChrisRackauckas/universal_differential_equations/blob/master/DelayLotkaVolterra/VolterraExp.jl

We plan to add this as a tutorial to the documentation soon, but it's this portion:

https://github.com/ChrisRackauckas/universal_differential_equations/blob/master/DelayLotkaVolterra/VolterraExp.jl#L93-L166

[aelmokadem]

That's great. Thanks Chris. I am really enjoying everything SciML is developing. Really intriguing work.

[ChrisRackauckas]

Leaving this open to mark that this should become a tutorial

[AlCap23]

Possible along with #157

[carlosmaria]

The code from the Universal Differential Equations paper is here: https://github.com/ChrisRackauckas/universal_differential_equations. Specifically, this is the example that first demonstrated it:

https://github.com/ChrisRackauckas/universal_differential_equations/blob/master/DelayLotkaVolterra/VolterraExp.jl

We plan to add this as a tutorial to the documentation soon, but it's this portion:

https://github.com/ChrisRackauckas/universal_differential_equations/blob/master/DelayLotkaVolterra/VolterraExp.jl#L93-L166

I would like to start doing some research on this area. Would you please update the last two links to the VolterraExp.jl examples? They are now broken. I would also appreciate any other example code similar to the one in section "2.3.1 Improved Identification of Nonlinear Interactions with Universal Ordinary Differential Equations" of your Universal Differential Equations paper. Thank you!

[ChrisRackauckas]

The documentation version is the one that's kept up to date: https://docs.sciml.ai/Overview/dev/showcase/missing_physics/. We should just start deleting things from that repo and start pointing to the docs.

[carlosmaria]

Thank you! I'll spend the winter break playing with this.

[ChrisRackauckas]

My plan for finishing our documentation overhaul (https://github.com/SciML/SciMLDocs/issues/107) includes updating those codes and making them into tested examples in the SciMLSensitivity documentation. If you're interested in helping update some of them, let me know.

For now I think this can close since it's out there in the docs.

[carlosmaria]

I'm flattered, but I am afraid I'm still too new to Julia to be useful. I had been following Kutz and Brunton's work, but I would like to try your Neural DE approach in fluid dynamics (free-surface flows) after I saw McKinley applications. I hope I will be able to contribute some simple work in that area.