ImportError: cannot import name 'DPOConfig' from 'trl'

[AswiniNLP]

from trl import DPOConfig I am not able to import DPOConfig, however, I can import DPOTrainer

can we use DPO on flanT5 , automodelforseq2seqLM ?

[younesbelkada]

Hi @AswiniNLP What is your TRL version? Can you try to install the latest TRL version pip install -U trl

[AswiniNLP]

Dear @younesbelkada, It is not working. Same error. TRL version is 0.8.6

[AswiniNLP]

Hi how to apply dpo on flanT5 ?

[trangtv57]

hey, the same issuse, I don't sure what version trl need to use so I can run dpo trainer script in example, so bad. tks

[AswiniNLP]

Dear, DPOconfig is not working

On Wed, 15 May 2024, 20:04 trangtv57, @.***> wrote:

hey, the same issuse, I don't sure what version trl need to use so I can run dpo trainer script in example, so bad. tks

— Reply to this email directly, view it on GitHub https://github.com/huggingface/trl/issues/1642#issuecomment-2112716640, or unsubscribe https://github.com/notifications/unsubscribe-auth/A7RYH2Y7H7K6D6QVW7TMTETZCNXA7AVCNFSM6AAAAABHVX3O56VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDCMJSG4YTMNRUGA . You are receiving this because you were mentioned.Message ID: @.***>

[AswiniNLP]

How to run dpo for flant5

[trangtv57]

install from source may help: pip install git+https://github.com/huggingface/trl.git

[AswiniNLP]

Tried not working

On Wed, 15 May 2024, 20:33 trangtv57, @.***> wrote:

install from source may help: pip install git+https://github.com/huggingface/trl.git

— Reply to this email directly, view it on GitHub https://github.com/huggingface/trl/issues/1642#issuecomment-2112797211, or unsubscribe https://github.com/notifications/unsubscribe-auth/A7RYH25L5BEJ7C75GOVGZ7LZCN2MVAVCNFSM6AAAAABHVX3O56VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDCMJSG44TOMRRGE . You are receiving this because you were mentioned.Message ID: @.***>

[AswiniNLP]

Can we use DPO on the top of flant5?

On Wed, 15 May 2024, 20:34 Aswini Kumar Padhi, < @.***> wrote:

Tried not working

On Wed, 15 May 2024, 20:33 trangtv57, @.***> wrote:

install from source may help: pip install git+https://github.com/huggingface/trl.git

— Reply to this email directly, view it on GitHub https://github.com/huggingface/trl/issues/1642#issuecomment-2112797211, or unsubscribe https://github.com/notifications/unsubscribe-auth/A7RYH25L5BEJ7C75GOVGZ7LZCN2MVAVCNFSM6AAAAABHVX3O56VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDCMJSG44TOMRRGE . You are receiving this because you were mentioned.Message ID: @.***>

[AswiniNLP]

Dear @younesbelkada, It is not working. Same error. TRL version is 0.8.6

Could you please help me know how to apply DPO on the seq to seq model through the DPO trainer?

[JhonDan1999]

the same issue here

[younesbelkada]

Hi there! I am sure you are facing a weird env conflict issue, make sure to use the latest TRL from pypi and refresh the kernels if you are using a google colab environment pip install -U trl

[JhonDan1999]

pip install -U trl

thank you @younesbelkada I already did !pip install --upgrade trl but the issue is still there

[HarryMayne]

install from source may help: pip install git+https://github.com/huggingface/trl.git

Installing 0.8.7 dev version from source works for me. Seems to work for 0.8.7 dev but not for 0.8.6. (latest release)

[younesbelkada]

let me try again and i will report back here

[younesbelkada]

Sorry for the confusion, indeed DPOConfig and SFTConfig are available features on main only, so as @HarryMayne pointed out, you need to install TRL from source. I will make a release soon on pypi to include that + many other bugfixes

[kevinjesse]

Installing from source on main does not resolve the missing DPOConfig. Even if DPOConfig.py is in trl/trl/trainer/, it is missing from the init.py.

[meixtan]

Installing from the source did not solve this for me. Same issue.

[github-actions[bot]]

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

[Rhitabrat]

not working