InternEvo icon indicating copy to clipboard operation
InternEvo copied to clipboard

Published 20 hours ago • verified publisher icon InternLM

[Feature] Partially frozen model support

Open scenarios opened this issue 2 years ago • 2 comments

Describe the feature

Current implementation does not support partially training, i.e., part of the model parameters are frozen. If I understand it correctly, the assertion around line 584 in hybrid_zero_optim.py suggests that all parameters should be involved in the training. image

Very looking forward this feature being implemented as there are many scenarios where users only want to finetune parts of the model.

Is there any plan on this feature?

Will you implement it?

  • [ ] I would like to implement this feature and create a PR!

scenarios avatar Aug 01 '23 07:08 scenarios

Sure, we will support freeze tuning ASAP.

SolenoidWGT avatar Aug 04 '23 12:08 SolenoidWGT

Sure, we will support freeze tuning ASAP.

Hi, any updates here?

scenarios avatar Sep 22 '23 04:09 scenarios