speed improvement

[exowanderer]

Using LMFIT to fit sine waves to Spitzer data, I can usually fit the model to 60k data points in ~15 seconds with leastsq as the method, and ~200 seconds with powell as the method.

With SPIDERMAN, using leastsq, it takes 650 seconds. I am still processing it with powell, but the point is that it's not even close to done yet.

Do you have any suggestions to speed up the fitting process? Do we have to create a new Parameters every single time (I tried not doing that). Is there a 'fast' version with less options?

Thank you

[exowanderer]

The powell method just finished after 17 hours (61677 seconds). Compared to 200s for simple, Fourier phase curve fits.

I want to use SPIDERMAN, but that speed is not useful.

[lkreidberg]

hey Jonathan -- yep, SPIDERMAN is definitely slow for Spitzer data. It's just expensive to compute an integral in 2D, no matter how clever you are about it.

To handle the issue of many data points, I typically calculate a SPIDERMAN lightcurve at ~100 points sampled uniformly in phase, and then interpolate the model to the observation times. This introduces small inaccuracies, but they're way below the uncertainty per data point

[exowanderer]

Excellent. Thank you.

I was debating about that, but I worried that down_sampling would lose too much resolution during ingress and egress. I just ran the same models with 1000 data points and it took 35 seconds (for the leastsq fitting procedure). The results seem to be wholly unchanged, which is great!

Thanks for the tip. That worked great

[exowanderer]

For anyone who wants to use this later, here is the end of my spiderman_model function call. This comes immediately after setting all of the spiderman_params:

from scipy.interpolate import CubicSpline

nspiders           = 1000
nskips             = times.size // nspiders
times_subsampled   = times[::nskips]
spider_lightcurve  = spider_params.lightcurve(times_subsampled)
spider_lightcurve  = CubicSpline(times_subsampled, spider_lightcurve)

spider_model = spider_lightcurve(times)

[lkreidberg]

Nice! In mine I actually also subsample at the first, second, third, and fourth contact, which improves the accuracy during ingress/egress.

This could be built into spiderman itself as a speed boost option - Tom, what do you think?