Monotonic constraints

[davidADSP]

An extremely useful feature of some gradient boosting libraries is the ability to enforce monotonic constraints - e.g.

https://xgboost.readthedocs.io/en/latest/tutorials/monotonic.html

Are there plans to get this implemented within NGBoost?

Thanks,

David

[alejandroschuler]

This would be very, very difficult to implement with any generality within ngboost because the mean prediction of a value may not map very nicely from a parameter of its distribution. For certain distributions it might be the case (e.g. Normal), but then we'd have to have all sorts of special hacks and whatnot that would work in some cases but not in others.

Furthermore, I'm proceeding under the assumption that what you want is monotonicity in terms of the expected value of the outcome, but that's only one interpretation. You might desire monotonicity in the median, or in 5% quantile, or in any other functional of the predicted distribution. All of these would be valid.

Point being: getting something like this in NGBoost would be an academic research project worthy of a paper in its own right, not a matter of implementing something that already exists. That makes it unlikely that it will ever be a feature, but especially at any point in the near future. It is, however, a great research question so if you know anyone working on a PhD who wants something interesting to work on you can send them this way!

[CDonnerer]

Hi @davidADSP , I've written an xgboost "version" of ngboost, which allows to use monotonic constraints, see example here. While I fully agree with the above caveats, at least for the special case of normal distribution and a (toy) dataset, it seemed to work. But will definitely need some testing in real life scenarios...

[astrogilda]

Hey Christian,

Really cool library! Related question: does xgboost-distribution offer a gpu implementation like xgboost, or nah? I'm assuming the relative performance numbers are for runs on the CPU, right?

On Sat, Aug 14, 2021, 5:39 AM Christian Donnerer @.***> wrote:

Hi @davidADSP https://github.com/davidADSP , I've written an xgboost "version" of ngboost, which allows to use monotonic constraints, see example here https://github.com/CDonnerer/xgboost-distribution#full-xgboost-features. While I fully agree with the above caveats, at least for the special case of normal distribution and a (toy) dataset, it seemed to work. But will definitely need some testing in real life scenarios...

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/stanfordmlgroup/ngboost/issues/188#issuecomment-898871837, or unsubscribe https://github.com/notifications/unsubscribe-auth/AFTOOHQNK75RSDL3IE6G7YTT4Y2WRANCNFSM4SKCXILA . Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&utm_campaign=notification-email .