[LightGBM] Regression: how to penalize negative predictions

[brunocous]

I have a simple regression task (using a LightGBMRegressor) where I want to penalize negative predictions more than positive ones. Is there a way to achieve this with the default regression LightGBM objectives (see https://lightgbm.readthedocs.io/en/latest/Parameters.html)? If not, is it somehow possible to define (many example for default LightGBM model) and pass a custom regression objective?

Thx in advance!

[welcome[bot]]

👋 Thanks for opening your first issue here! If you're reporting a 🐞 bug, please make sure you include steps to reproduce it.

[tyler-romero]

The simplest way might be to duplicate positive examples in your training set so that they naturally carry more influence during training.

[imatiach-msft]

"define (many example for default LightGBM model) and pass a custom regression objective" @brunocous unfortunately, we don't have this currently in the mmlspark wrapper of lightgbm, but it would definitely be a great feature to add at some point.

[imatiach-msft]

I should update this thread that custom objectives have been added recently, so this should be possible now

[DefUs3r]

Hi @imatiach-msft could you let me know if I can work on this?

[imatiach-msft]

@DefUs3r sure, what do you want to work on specifically? Are you thinking of adding an example on how to penalize negative predictions using a custom loss function?

[Mihir-Khandelwal]

Hii @brunocous everything is almost fine on your provided link, but it could be more Attractive by using some more UI features, color schemes, and images. Also in your Python code learning, you can give the idea for them starting from the basics.

[imatiach-msft]

@Mihir-Khandelwal are you commenting on the right thread? I think you might have meant to comment elsewhere?

[pragyasrivastava0805]

Can I be assigned this @imatiach-msft ?

[imatiach-msft]

@pragyasrivastava0805 sure, I just assigned the task to you

[pragyasrivastava0805]

@imatiach-msft i am very new to open source,could you please walk me through and tell what exactly needs to be done

[Mihir-Khandelwal]

@imatiach-msft Sorry i am new to this, as an microsoft recruting task, i was assigned a task to comment a code correction on any of the 100+ stars github account.

[navya002]

I have a simple regression task (using a LightGBMRegressor) where I want to penalize negative predictions more than positive ones. Is there a way to achieve this with the default regression LightGBM objectives (see https://lightgbm.readthedocs.io/en/latest/Parameters.html)? If not, is it somehow possible to define (many example for default LightGBM model) and pass a custom regression objective?

Thx in advance!

I want to ask just for clarity purpose that here ,positive and negatives samples refer to overestimation and underestimation , correct?

[brunocous]

I want to ask just for clarity purpose that here ,positive and negatives samples refer to overestimation and underestimation , correct?

Correct. In the end, I went with just using the quantile regression objective to penalise overestimations more. That worked well enough for my use case. Second trick that helped a lot is assigning samples that are more prone to overestimation a higher sample weight for training. Not perfect solutions, but it does the trick.

[navya002]

Hi ,

Thanks for responding , that resonates with me as well. Also, while I was looking at it (the problem) I optimised objective function a bit for better results since in the 50th percent quantile it turns out to be mae , I changed it a bit for better results.Please have a look and let me know what you think (I have submitted the pull request with that function).

Hoping it helps and awaiting your reply.

Thankfully Navya

On Mon, Aug 30, 2021, 18:01 brunocous @.***> wrote:

I have a simple regression task (using a LightGBMRegressor) where I want to penalize negative predictions more than positive ones. Is there a way to achieve this with the default regression LightGBM objectives (see https://lightgbm.readthedocs.io/en/latest/Parameters.html)? If not, is it somehow possible to define (many example for default LightGBM model) and pass a custom regression objective? Thx in advance!

I want to ask just for clarity purpose that here ,positive and negatives samples refer to overestimation and underestimation , correct?

Correct. In the end, I went with just using the quantile regression objective to penalise overestimations more. That worked enough for my task at hand. Second trick that helped a lot is assigning samples more prone to overestimate a higher sample weight. Not perfect solutions, but it does the trick.

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/microsoft/SynapseML/issues/918#issuecomment-908303142, or unsubscribe https://github.com/notifications/unsubscribe-auth/ALH4H56VPQRNJKBSBFWUF53T7N22XANCNFSM4QS2APIA . Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.

[karnvaibhav11]

I want to ask just for clarity purpose that here ,positive and negatives samples refer to overestimation and underestimation , correct?

Correct. In the end, I went with just using the quantile regression objective to penalise overestimations more. That worked well enough for my use case. Second trick that helped a lot is assigning samples more prone to overestimate a higher sample weight for training. Not perfect solutions, but it does the trick.

Please assign me this @brunocous @imatiach-msft

[karnvaibhav11]

Hi, I have found a way to penalize the negative prediction in Regression by using the concepts of Adaboost penalization of wrong Classification Penalization. Basically what it does is, it updates the weight of the negative residual and increase its training example for the next model. This ensures that if the value is overestimated it penalizes it more. It's a completely new concept which I researched and the work is still in progress.
Please find the attached Docx. which has the steps that I followed and pdf for the code. Note that I haven't created the intervals due to the lack of time and to meet my Microsoft Project deadline. But I'll work on it later on. Also, I am unable to create a pull request so I am sharing these files over here. Will love to hear your feedback. Thanks.

Penalizing the Negative Prediction in a Regression Problem.docx M2LinearRegression.pdf

[ParulSrivastava]

Penalizing.the.Negative.Prediction.in.a.Regression.Problem.docx

I have Created a pipeline using Azure ML for Regression Model