Mini-batching

[paulflang]

I am working with a model for which I expect experimental data under multiple perturbed conditions. To speed up the fitting procedure, I was wondering if there are any plans to implement mini-batching in pyPESTO.

[paulstapor]

Not directly planned in the next time (this is what we had parPE for)... However, implementing a rough version of such an optimizer would not be super-complicated. We had that in Matlab-Pesto (DeLOS) as a proof-of-concept back then... Recoding this in Python should be mostly straight-forward... An alternative might be (don't know how easy it would be) to interface some existing packages such as Tensorflow and its optimization routines. However, I think that's maybe not the best for for ODE models...

[paulstapor]

Back here @paulflang : I was thinking about the mini-batch optimization... If you know any python package, which implements mini-batch optimizers with black-box objective functions (i.e., without assuming a neural net structure), we could try to link that to pyPESTO in a first attempt... Would prefer something implementing Adam and possibly also SAG-type things...

[FFroehlich]

Tensorflow can do that, just needs an interface to the objective.

[FFroehlich]

Tensorflow can do that, just needs an interface to the objective.

Which we will need anyway for pymc4

[paulflang]

Thanks for thinking about that.

If you know any python package, which implements mini-batch optimizers with black-box objective functions (i.e., without assuming a neural net structure), we could try to link that to pyPESTO in a first attempt...

I am afraid, I am not familiar with Python packages that implement mini-batching.