Adding models that don't require a derivative

[ironictoo]

This is an example model that shows how new models can be easily added without the need for an explicit derivative.

A numerical derivative is instead used, I included both a 3 point and 5 point version. In my use there was no significant difference between the two so I used the 3 point as the performance should be marginally better.

Adding this example or a similar one can gives users an easy option for models where the derivative can be difficult to calculate, or just cumbersome to input without errors.

[jkfindeisen]

I think this is a good idea.

[jkfindeisen]

I looked a bit more at the example and it seems to be a model for a 1D convolution with an exponential? Not sure. One would also need a bit of documentation about the model and maybe it should be one where it actually makes sense to use numerical derivatives, because they are otherwise difficult to compute. Here it might be simple to actually have an analytical solution for the derivatives.

Also the question is a bit how we want to support custom models. See especially #37. It would be good to make some progress there so new model PRs are easier to decide.

If it should only be an example model, currently one cannot add models without performance overheads. This means including it would affect performance in production. Therefore I'm currently undecided about this.

The ideal solution would be something like:

• users can specify their own models on the fly (without needing to recompile Gpufit) and without performance overhead
• these custom models can specify whether a derivative is supplied or not
• if no derivative is supplied, it's approximated numerically within the framework

[jimkring]

@jkfindeisen I agree with your points about an ideal solution. It might be useful to check out the jaxfit library, if you haven't seen it.  It uses automatic differentiation to calculate the Jacobians (instead of requiring partial derivatives or implementing numeric approximation), and does just-in-time (JIT) compilation of the model to allow "on the fly" changes to the user's model (written in Python code). Gpufit appears to perform much better for fitting smaller data sets (see screenshot below) and it would be interesting to see Gpufit could provide some of the flexibility of jaxfit without giving up computational performance.

Below is some benchmarking charts I found in a paper online...

[superchromix]

In the short term, I agree it would make sense to include an example model in which the derivatives are computed numerically. Even if the model is trivial with analytic derivatives, this would be useful for users.

@jimkring I will look into Jaxfit - that's an interesting development. Thanks.

[superchromix]

Regarding Jaxfit, I'd note that the package is designed to compute a single fit operation in each call, whereas Gpufit is designed for the parallel execution of large numbers of fits. Also, the smallest data size per fit which they tested is 10^4, which seems relatively large. Gpufit is not designed/optimized for calculation of a single fit with a large data size.