Patrick Kidger

Results 267 comments of Patrick Kidger

Thanks :) Yeah, that's my impression. (FWIW I've tried mkdocstrings but haven't managed to get it working.) Now I'm considering rolling my own... :D

Hi there! Glad you like it. So the traffic here is low-volume enough that I'm happy to field these kinds of queries just via issues, rather than creating a whole...

It looks like your tensor `x` has the wrong shape. As per [here](https://github.com/patrick-kidger/torchcde#constructing-controls), the tensor must have shape `(..., length, channels)` -- in particular it is the penultimate dimension that...

There's two main possibilities here. The first is that your dynamics are really stiff, or otherwise maladapted to your choice of numerical solver. Trying adjusting tolerances, changing the integration method,...

Right. So via a debugger or otherwise, try and track down where that nan is coming from. Is it a nan arising from e.g. a division by zero, or is...

Probably you're doing something like using a neural network vector field without a tanh at the end. Generally speaking you want to constrain the rate of change of hidden state,...

To be clear, I don't suggest making all activations `tanh`. Rather, parameterising your vector field as something like `tanh(mlp(...))`, where `mlp` is some MLP using e.g. a softplus activation. It's...

Have a read of Chapter 4 of [On Neural Differential Equations](https://arxiv.org/abs/2202.02435).

It doesn't look like you're specifying the solver, or the tolerances/step-sizes at all. The defaults may not be suitable for your data. You may like to try [Diffrax](https://github.com/patrick-kidger/diffrax), which is...