Mert Sayar
Results
2
issues of
Mert Sayar
Start index under `batched_forward_pass` starts from the last query token which does not align with the comment in line 1032. This also causes a problem when handling the response tokens...
The code and the comment do not align in: https://github.com/huggingface/trl/blob/b68ff96f0c74368961e194081e122959cd1f4d4d/trl/trainer/ppo_trainer.py#L1032 In the comment, it is stated that logprobs starts from the second query token, however `start` starts from the last...