ValueError with the LEAR model when using scikit-learn 1.1.0 or newer

[ikonentj]

Hi,

Starting from version 1.1.0, scikit-learn class LassoLarsIC takes the keyword argument noise_variance. This causes an error with LEAR model if the calibration window is set to a relatively small value (e.g., 84 days, which is used in the paper). Here are the commands to reproduce the error:

conda create -n epf_env python=3.9
conda activate epf_env
git clone https://github.com/jeslago/epftoolbox.git
cd epftoolbox
pip install .
cd examples
python recalibrating_lear_flexible.py --dataset PJM --years_test 2 --calibration_window 84

ValueError: You are using LassoLarsIC in the case where the number of samples is smaller than the number of features. In this setting, getting a good estimate for the variance of the noise is not possible. Provide an estimate of the noise variance in the constructor.

I obtained the above error with scikit-learn version 1.1.3. When I use scikit-learn 1.0.1, the LEAR model works fine.

[jeslago]

@gmarcjasz What do you suggest we do? Can we provide a reasonable noise_variance? Or should we instead limit the version of scikit-learn?

Limiting the version of scikit-learn to 1.0.1 might be worse in the long term.