Built-in R2 (R-squared) metric

[jameslamb]

Description

LightGBM should add a built-in R^2 metric.

Benefits of this work

• prevents the need to implement this metric in a custom function (as seen in #6982, for example)

Acceptance criteria

• LightGBM supports an r2 metric (on both CPU and GPU)

Approach

Do something like this to find where rmse shows up:

git grep -i rmse

And follow that as a guide for what to update.

At a minimum:

• https://github.com/microsoft/LightGBM/blob/master/src/metric/regression_metric.hpp
• https://github.com/microsoft/LightGBM/blob/master/src/metric/cuda/cuda_regression_metric.cpp
• https://github.com/microsoft/LightGBM/blob/27dcf820f30428a5e74e52e6fa0dbb271316f899/include/LightGBM/config.h#L1013
• https://github.com/microsoft/LightGBM/blob/27dcf820f30428a5e74e52e6fa0dbb271316f899/R-package/R/metrics.R#L9

Notes

This should be the standard R^2, not the "adjusted" R^2 which penalizes models with more features.

See https://en.wikipedia.org/wiki/Coefficient_of_determination

[jameslamb]

I've added this to #2302 along with other feature requests.

Adding the awaiting response label so it'll automatically be closed after 30 days without any comments. Anyone finding this is welcome to implement it... just leave a comment saying you'd like to work on it.

[nicklamiller]

Hi @jameslamb, I'd like to work on this issue, I've done a little preliminary work so should have a PR open soon-ish.

[jameslamb]

Sure, that's be great @nicklamiller ! Thanks!

[nicklamiller]

@jameslamb sure thing, I've implemented the CPU version of R-squared in #7008. As mentioned in that PR, I skipped the CUDA implementation but planned to add it in a follow-up PR as part of this issue.

Part of why I didn't include it in #7008 is because, right now, it looks like the CUDA logic for metrics are pointwise focused (the only CUDA file I could see for metrics is src/metric/cuda/cuda_pointwise_metric.cu), so adding support for something like R^2 that doesn't really have a pointwise implementation seemed like it may be a little more involved.

I'll think through that a bit more, though, and offer a specific implementation in the follow-up PR.

[jameslamb]

Awesome, thank you! That makes sense to me. There are plenty of other metrics we don't yet have CUDA support for: https://github.com/microsoft/LightGBM/issues/5163#issuecomment-3153143722

I noticed #7008 is still in draft and its CI is failing... I won't review there until you tell me it's ready (either by @-ing me or by taking it out of draft). Let me know any time if you need help, and thanks for doing this 😁