Open-Assistant icon indicating copy to clipboard operation
Open-Assistant copied to clipboard

accelerator version issue

Open idisuu opened this issue 2 years ago • 2 comments

huggingface's accelerator updated from v0.19.0 to v0.20.0 and 'logging_dir' disappeared from init method in Accelerator class.

So the above error occurs.

image https://github.com/huggingface/accelerate/blob/baebae3bbecbea05d721a50917f352cccd14811e/src/accelerate/accelerator.py#L242-L244

OA doesn't specify version https://github.com/LAION-AI/Open-Assistant/blob/0fcf3e08fe62295d4696e590005b0f33383342ea/model/pyproject.toml#L12-L13

However, if you go to the trlx library that actually runs the accelerator, it is versioned as shown below. image https://github.com/CarperAI/trlx/blob/0dce99d96b7d70b6a9114129d8e38bf6c80eb653/requirements.txt#L1-L2

Of course, it is true that the trlx library also has its own errors.

However, if OA will have a dependency on the trlx library, I think it is necessary to get the trlx requirement.txt and install it with the same version as specified there.

idisuu avatar Jun 08 '23 02:06 idisuu

Could you pleas provide the command you were running?

CloseChoice avatar Jun 17 '23 14:06 CloseChoice

Could you pleas provide the command you were running?

  1. Create a new virtual environment using Anaconda
  2. Command pip install -e . in Open-Assistant/model folder, So dependencies can be installed according to pyproject.toml file
  3. Run trainer.py file. I used my own script. But I think example code provided in Open-Assistant/model/model_training would make the same error. python trainer_rl.py --configs defaults_rlhf --cache_dir $DATA_PATH --rank_model $REWARD_MODEL --sft_model $SFT_MODEL --output_dir $MODEL_PATH/rl_model
  4. Because error is occured when calling trainer.py - trlx.train method https://github.com/LAION-AI/Open-Assistant/blob/8ea3f1464db0c18c4311a68275d28692660abc23/model/model_training/trainer_rl.py#L184-L191
  5. trlx library uses HuggingFace's accelerator library when training RLHF using PPO.
  6. And trlx library manage their own accelerator version. (For example now is 0.18.0)
  7. But Open-Assistant's accelerator version (installs latest version automatically, now is 0.20.0) is not compatible with trlx code that uses accelerator.
  8. To summarize, Open-Assistant uses trlx and trlx uses accelerator. trlx use older version of accelerator, but Open-Assistant's dependency file installs latest version of accelerator. So error occurs when using trlx

idisuu avatar Jun 19 '23 01:06 idisuu