Density weighted linear regression

[loicdtx]

A single "winter outlier" can sometimes have a large influence on the model fit. Winter observation are often less dense and of lower quality; reducing their weight in the regression could help having more stable models. Implementing density weighted linear regression as a fit_method could be useful.

How?

• Compute density using Kernel Density Estimation
• Use density as weight in the regression

Pseudo code:

import numpy as np
from scipy.stats import gaussian_kde

kde = gaussian_kde(X, bw_method=0.5)
density = kde(X)

W = np.diag(density)
XTWX = np.dot(np.dot(X.T, W), X)
XTWy = np.dot(np.dot(X.T, W), y)
beta = np.linalg.solve(XTWX, XTWy)