Wang, Yi comments

Results 69 comments of


                                            Wang, Yi

add accelerate example for DDP and FSDP in sequence classification fo…

@pacman100 please help review.

add accelerate example for DDP and FSDP in sequence classification fo…

yes, @pacman100 I see memory decrease in FSDP. I finetune llama 7b with 2-GPUs (RTX8000) using p-tuning, if FSDP is not used, DDP will be crashed because of OOM if...

fix the issue that ipex is always imported if it's installed in python env.

@abhilash1910 @muellerzr please help review

fix the issue that ipex is always imported if it's installed in python env.

@mingxiaoh

fix the issue that ipex is always imported if it's installed in python env.

@muellerzr @sgugger I have updated the PR according to the discussion, please help review it.

enable prompt tuning/prefix tuning/p tuning clm and example

@libinta @regisss please help review. prompt tunning is just a first step, next I will enable prefix-tuning and ptuning based on the example

enable prompt tuning/prefix tuning/p tuning clm and example

prompt tunning, prefix-tuning and ptuning is enabled for llama finetune and inference

enable prompt tuning/prefix tuning/p tuning clm and example

inference part needs model change to adapt to prompt tuning/prefix tuning, since virtual token is used in prompt/p tuning and past_key_value is extended in prefix tuning.

enable prompt tuning/prefix tuning/p tuning clm and example

also upgrade the peft to latest tag 0.9.0

enable prompt tuning/prefix tuning/p tuning clm and example

prompt tunning+deepspeed zero3 issue fixed by https://github.com/huggingface/peft/pull/1591