experiments_with_python
experiments_with_python copied to clipboard
Using scipy's genetic algorithm for initial parameter estimation in non-linear regression
I see on github that you are writing optimization code using Python. When using non-linear solvers such as Levenberg-Marquardt or gradient descent to optimize parameters for non-linear regression, there is a general problem of initial starting parameters. With poor initial parameter values to start the "descent" into error space, the algorithms can stop in a local error minimum. For this reason, the authors of scipy have added a genetic algorithm for initial parameter estimation to use with scipy's non-linear solvers. The module is named scipy.optimize.differential_evolution. Scipy's implementation uses the Latin Hypercube algorithm to ensure a thorough search of parameter space.
I have used scipy's Differential Evolution genetic algorithm to determine initial parameters for fitting a double Lorentzian peak equation to Raman spectroscopy data of carbon nanotubes and found that the results were excellent. The GitHub project, with a test spectroscopy data file, is:
https://github.com/zunzun/RamanSpectroscopyFit
My background is in nuclear engineering and industrial radiation physics, and I love Python, so if you have any questions please let me know.
James Phillips
Hi James,
Thanks for your post! I'll check out SciPy's differential evolution for initial parameter estimation. In the past, I've used Bayesian Optimization that utilizes Gaussian Process to explore parameter space in a way that trades off exploration and exploitation. The goal is to minimize the number of parameter queries in the case where it's expensive to evaluate performance for a given set of initial parameters. Spearmint is another popular library for Bayesian Optimization.
Vadim
Cool. I had not heard of Spearmint.