Add a new reward model and make some modifications to reward-bench code

[kirigayahitsugi]

Hi RewardBench Team,

We have updated a 8B reward model (Custom Classifier) general-preference/GPM-Llama-3.1-8B and a 2b reward model (Custom Classifier) general-preference/GPM-Gemma-2B.

Local evaluation results for our models are listed as bellow:

For general-preference/GPM-Llama-3.1-8B: {'Chat': 0.9329608938547486, 'Chat Hard': 0.8859649122807017, 'Safety': 0.9055003159003159, 'Reasoning': 0.9597485949691711}

For general-preference/GPM-Gemma-2B: {'Chat': 0.7150837988826816, 'Chat Hard': 0.6973684210526315, 'Safety': 0.810949104949105, 'Reasoning': 0.7550369673159819}

We have made some modifications to RewardBench code (v0.1.2), which includes:

1. Incorporate a custom RewardBench pipeline.
2. Create a tailored CustomRewardModel class along with a function to load our model.
3. Implement a custom method for calculating results (scores) for our General Preference model.
4. Introduce additional custom arguments essential for our model.

We have integrated code adapted from RewardBench (v0.1.2) into our repository, general-preference-model, specifically under rewardbench_eval directory.

I would like to inquire about the best practices for incorporating RewardBench with our modifications into the repository. Additionally, we are interested in adding this new reward model to the RewardBench Leaderboard.

Thank you for your time and help!

Best regards, Grace Zhang

[natolambert]

Hey @kirigayahitsugi, there's really similar discussion to this in #198 -- let me know if you need more information. There is a brief instruction on adding custom pipelines here: https://github.com/allenai/reward-bench/tree/main/rewardbench/models

[kirigayahitsugi]

Thank you for your response. May I modify the pipeline-related components in my code following the instruction you mentioned and submit both those changes and any existing modifications to a new feature branch in a PR?

[natolambert]

@kirigayahitsugi - yes that's the normal process!

[kirigayahitsugi]

Thank you for your reply!