MiniCPM-V
MiniCPM-V copied to clipboard
WSD scheduler
Can I apply wsd scheduler in This repo's finetuning code?
When i set as follows,
--lr_scheduler_type warmup_stable_decay \
i got this error
Traceback (most recent call last):
File "/home/raid/repos/ac4aa168-96f1-47ab-9361-1c15cb5c48cf/MiniCPM-V/train.py", line 392, in <module>
train()
File "/home/raid/repos/ac4aa168-96f1-47ab-9361-1c15cb5c48cf/MiniCPM-V/train.py", line 373, in train
trainer.train(resume_from_checkpoint=None)
File "/usr/local/lib/python3.10/dist-packages/transformers/trainer.py", line 1885, in train
return inner_training_loop(
File "/usr/local/lib/python3.10/dist-packages/transformers/trainer.py", line 2045, in _inner_training_loop
model, self.optimizer, self.lr_scheduler = self.accelerator.prepare(
File "/usr/local/lib/python3.10/dist-packages/accelerate/accelerator.py", line 1291, in prepare
result = self._prepare_deepspeed(*args)
File "/usr/local/lib/python3.10/dist-packages/accelerate/accelerator.py", line 1758, in _prepare_deepspeed
engine, optimizer, _, lr_scheduler = deepspeed.initialize(**kwargs)
File "/usr/local/lib/python3.10/dist-packages/deepspeed/__init__.py", line 181, in initialize
engine = DeepSpeedEngine(args=args,
File "/usr/local/lib/python3.10/dist-packages/deepspeed/runtime/engine.py", line 307, in __init__
self._configure_lr_scheduler(lr_scheduler)
File "/usr/local/lib/python3.10/dist-packages/deepspeed/runtime/engine.py", line 915, in _configure_lr_scheduler
self.lr_scheduler = client_lr_scheduler(self.basic_optimizer)
File "/usr/local/lib/python3.10/dist-packages/transformers/integrations/deepspeed.py", line 351, in _lr_scheduler_callable
lr_scheduler = trainer_copy.create_scheduler(
File "/usr/local/lib/python3.10/dist-packages/transformers/trainer.py", line 1398, in create_scheduler
self.lr_scheduler = get_scheduler(
File "/usr/local/lib/python3.10/dist-packages/transformers/optimization.py", line 547, in get_scheduler
return schedule_func(
TypeError: get_wsd_schedule() got an unexpected keyword argument 'num_training_steps'