Feature Request: rotated PCA

[raybellwaves]

To reproduce the methodology CPC use to calculate the NAO e.g http://www.cpc.ncep.noaa.gov/data/teledoc/telepatcalc.shtml

I see they first calculate the 10 leading eofs for each month (3 month average) which this code can do as it stands. They then apply a ‘Varimax rotation’ (I need to read their paper) to obtain the 10 rotated eofs for each month (3 month average)

[ajdawson]

You can try the very old and undocumented experimental-rotation branch. I don’t plan on adding this feature myself unless someone else does all the work.

[raybellwaves]

I’ll take a look. Thanks. Hopefully I have time work on this

[mariajmolina]

@raybellwaves Did you happen to figure out how to apply the 'Varimax rotation'? I am wondering how to do this very same thing for computing the NAO index.

[raybellwaves]

@hurricanemolina afraid I haven't spent time on this yet. I've got as far as doing some background reading https://github.com/raybellwaves/eofs/blob/rotation/lib/eofs/rotation/standard.py @ajdawson's experimental-rotation branch isn't in too bad shape. i.e. he does the varimax rotation in https://github.com/ajdawson/eofs/blob/experimental-rotation/lib/eofs/experimental/rotation/kernels.py

[raybellwaves]

(I think) i've worked out how to do it. See https://github.com/raybellwaves/rot-eof-dev and reof.ipynb. rot-eof-cpc_old.ipynb was my first attempt and has lots of background reading. I used the experimental-rotation branch in this package and https://github.com/bmcmenamin/fa_kit/blob/master/fa_kit/rotation.py

The results look good when compared to the method in NCL (plots included).

I can't explain the https://github.com/bmcmenamin/fa_kit/blob/148700747720211007307434eb398c1d4ce60990/fa_kit/rotation.py#L35 function. But it is equivalent to the NCL function eofunc_varimax_reorder function.

Would someone mind reviewing the ipython notebook for my calculation and provide better documentation for the flip function. Once @ajdawson is happy with this I'll start work on a PR

[raybellwaves]

Knocking this on the head. The spatial patterns don't quite match the results in NCL and I can't work out why. Anyone is welcome to fork the repo and give this a stab. Sad to say that i'll be using NCL for this application.

[nicrie]

Would this still be a requested feature today? I just created my own package (for performing Maximum Covariance Analysis) including Varimax and Promax rotation, so I could give it a try.

[JulianGiles]

@nicrie I would really appreciate if this feature gets deployed. I haven't found another EOF Python package that is as clean as this one, so I think it would be a nice addition.

[nicrie]

I had a look at the experimental-rotation branch and it indeed looks quite good. The design of using a rotator class makes sense to me, and the calculations are accurate. What's still missing is the reordering of the rotated EOFs according to their explained variance.

@raybellwaves I checked against the example of NAO calculated by CPC and the results match.

I'll implement it later today + add the xarray interface. @ajdawson are there any guidelines to follow for PRs? It's the first time that I contribute to another project, some information would be very helpful.

[nicrie]

@ajdawson I just opened a PR adding Varimax-rotated PCA functionality to the standard interface. Currently implemented for rotated PCA is:

• eofs
• pcs
• eigenvalues (representing variance of each mode after rotation)
• varianceFraction
• eofsAsCorrelation
• eofsAsCovariance
• reconstructedField using the rotated modes
• projectField: project a new field on rotated EOFs to obtain rotated pseudo-PCs

I did not implement northTest given that the rule of thumb does not hold anymore after rotation.

What's still missing:

• test cases
• interface for xarray, cdms and iris

I can provide test cases as well as interface for xarray though not for cdms or iris since I have never worked with these modules myself.

---

Side note: While browsing the code I found that projectField in the standard interface does not take into account the centering of the input field . From #99 I see that you did this deliberately but it seems a bit counter-intuitive to me. In particular when you project new, unseen data onto the EOFs it would be nice to use the internally stored mean field. In case you're open for this, I'd happily make another PR.

[nicrie]

In case anyone is interested: Varimax and Promax rotated EOF analysis is now available in xeofs.