Idan Shenfeld

Add SDF body and explicit integrators. Code is taken from https://github.com/eanswer/RoboticHandDesign/commit/8bf0b9ccb4c737f0c145471604ad573b5d9ab330

Bug in calling model.eval() in PPO

3

When performing PPO step, the code perform the forward pass in [line 798](https://github.com/huggingface/trl/blob/main/trl/trainer/ppo_trainer.py) using the function "batched_forward_pass". However, "batched_forward_pass" put the model in eval mode ([line 986](https://github.com/huggingface/trl/blob/6d1cb85e73e990c5befac09988528996e8261080/trl/trainer/ppo_trainer.py#L986C1-L986C21)): model.eval() I'm pretty...

This PR introduces the implementation of Value Augmented Sampling (VAS), based on the paper [Value Augmented Sampling for Language Model Alignment and Personalization](https://arxiv.org/abs/2405.06639). VAS enhances language model generation by incorporating...