Promote AveragingLearner to LinearFitLearner

[jbweston]

(original issue on GitLab)

opened by Anton Akhmerov (@anton-akhmerov) at 2017-09-06T09:31:46.601Z

A more advanced version of averaging would be performing a linear fit. The logic here should be largely similar, but instead of using the mean, we should use fit coefficients.

[jbweston]

originally posted by Bas Nijholt (@basnijholt) at 2018-05-29T19:42:12.834Z on GitLab

@anton-akhmerov I could really use this right now.

Could you perhaps elaborate on your ideas of how the LinearFitLearner would work?

[jbweston]

originally posted by Anton Akhmerov (@anton-akhmerov) at 2018-05-29T20:19:23.515Z on GitLab

You basically need to incrementally update this formula, so you need to accumulate a matrix of $\sum x_i x_i^T$ and $\sum x_i y_i^T$. For the error estimate you would need to store higher moments as well.