trl icon indicating copy to clipboard operation
trl copied to clipboard

Published 20 hours ago • verified publisher icon huggingface

Neftune is applied twice; in trl and transformers BOTH!

Open MilkClouds opened this issue 8 months ago • 1 comments

As long as I read is right, neftune is being applied twice both in trl and transformers.

https://github.com/huggingface/trl/blob/39a7d1c121d26224fd7455d3d2038e0d20831c54/trl/trainer/sft_trainer.py#L440 calls Trainer.train and Trainer.train have self.model = self._activate_neftune(self.model) inside.

Please let me know if I misunderstood.

Releated PR

  • https://github.com/huggingface/trl/pull/871

MilkClouds avatar Jun 24 '24 02:06 MilkClouds