[Bug] Spectral mixture kernel implementation

[pipme]

🐛 Bug

The forward() function's implementation in gpytorch/gpytorch/kernels/spectral_mixture_kernel.py seems to be wrong. I am also wondering which formula you are using. @gpleiss

1. The formula (12) in this paper is uncorrected. The correction can be found in https://www.cs.cmu.edu/~andrewgw/typo.pdf. Current implementation seems to follow this uncorrected formula.
2. In current implementation, the product and summation are in wrong order. It should compute product first and then perform summation over mixture components.

[patel-zeel]

Hey, @pipme and @gpleiss. Would you like to fix this with a PR or shall I give it an attempt?

[pipme]

@patel-zeel I have a fixed version and may be able to make a PR later this week after some testing with batches etc.

[patel-zeel]

Sounds great @pipme. I was wondering if you could help me with a problem related to this kernel (forecasting Mauna loa co2). I have added the details in this discussion.

[gpleiss]

@pipme sorry for the slow reply! We'd definitely welcome a PR, but it would have to be backwards compatible (otherwise it would be a breaking change). I think the thing to do is:

• Create a boolean flag correct_multidimensional_mixture in the SMKernel constructor (default: False). Use the correct computations if True, and use the current code otherwise.
• Throw a deprecation warning if correct_multidimensional_mixture is False.
• Update the SM unit tests to test both behaviors
• Update the SM docs to describe both behaviors