Surprise icon indicating copy to clipboard operation
Surprise copied to clipboard

Added asym_rmse and asym_mae accuracy measures

Open martincousi opened this issue 6 years ago • 4 comments

Hi, I have created two new accuracy measures for a project I am working on. These measures are asymmetric and can put more emphasis on lower or higher ratings. They are useful when the rating distribution is asymmetric. See [1] for more details.

I don't know how to include these measures in GridSearchCV since they require a weight parameter.

P.S. This is my first time contributing to a library so I am open to suggestions on how to improve.

[1] R. Zhu, D. Niu, L. Kong, and Z. Li, “Expectile Matrix Factorization for Skewed Data Analysis,” in Proceedings of the 31th Conference on Artificial Intelligence (AAAI 2017), 2017, pp. 259–265.

martincousi avatar Mar 26 '18 21:03 martincousi

Thanks for the PR and for the details :)

Can you please fix the pep8 issue?

I don't know how to include these measures in GridSearchCV since they require a weight parameter.

Yes that's a good point. I guess we'll have to implement a more sophisticated way to pass the measures parameter in GridSearchCV and in cross_validate. I haven't gone through the details but I think the way to go with scikit-learn is to use make_scorer, so we could make something like that.

NicolasHug avatar Mar 29 '18 11:03 NicolasHug

I think that I solved all pep8 issues now.

martincousi avatar Mar 29 '18 11:03 martincousi

Thanks,

I'll leave the PR open and merge it when we find a way to address the previous point. I'd rather not merge it now as it's not fully compatible with GridSearchCV and cross_validate.

NicolasHug avatar Apr 02 '18 12:04 NicolasHug

Ok, if I will see if I get the time to adjust the way a measure is passed.

martincousi avatar Apr 05 '18 17:04 martincousi