stanford_alpaca icon indicating copy to clipboard operation
stanford_alpaca copied to clipboard

Published 20 hours ago • verified publisher icon tatsu-lab

forward prefetch error: transformers version

Open sallywang147 opened this issue 2 years ago • 12 comments

Hi,

thanks for your great work! which transformers did you use for training? I tried to reproduce the result with transformers 4.28.0, but I got the following error: __init__() got an unexpected keyword argument 'forward_prefetch' in transformers/trainer.py. Thank you for your help.

sallywang147 avatar Mar 21 '23 11:03 sallywang147

same error

447428054 avatar Mar 23 '23 01:03 447428054

same error

TypeError: init() got an unexpected keyword argument 'forward_prefetch'

@sallywang147 @447428054 Are you fix the error?

Jacksonlark avatar Mar 23 '23 04:03 Jacksonlark

May be torch version is not compatible with transformers

https://pytorch.org/docs/1.12/fsdp.html

pytorch version <= 1.12, The FullyShardedDataParallel not have forward_prefetch.

Jacksonlark avatar Mar 23 '23 05:03 Jacksonlark

@Jacksonlark @sallywang147 i fix it by installing transformers==4.26.1, I'll post the code in a moment

447428054 avatar Mar 23 '23 05:03 447428054

@Jacksonlark @sallywang147 i fix it by installing transformers==4.26.1, I'll post the code in a moment

@447428054 Using transformers==4.26.1, How do you load llama model?

Jacksonlark avatar Mar 23 '23 05:03 Jacksonlark

@447428054 yeah same question as @Jacksonlark. I can't load the model without the latest transformer

sallywang147 avatar Mar 23 '23 15:03 sallywang147

@447428054 yeah same question as @Jacksonlark. I can't load the model without the latest transformer

You can check new branch from transformers==4.26.1 and migration llama code, then pip this branch.

Jacksonlark avatar Mar 24 '23 03:03 Jacksonlark

Ah thanks! Actually switching torch version worked for me. I still used the latest transformer 4.28.0dev0.

sallywang147 avatar Mar 24 '23 03:03 sallywang147

@sallywang147 How are you getting it to fine-tune? My GPU's seem to freeze up after a few minutes, I see no communication or usage, just memory usage and my entire pipeline seems stuck

srikanthsrnvs avatar Mar 24 '23 03:03 srikanthsrnvs

@447428054 yeah same question as @Jacksonlark. I can't load the model without the latest transformer

You can check new branch from transformers==4.26.1 and migration llama code, then pip this branch.

Yeah, The cod is as follows:https://github.com/447428054/stanford_alpaca_llama.git

447428054 avatar Mar 25 '23 07:03 447428054

Ah thanks! Actually switching torch version worked for me. I still used the latest transformer 4.28.0dev0.

what version of torch is used by you?

xiaoweiweixiao avatar Mar 29 '23 12:03 xiaoweiweixiao

Same question. What version of torch?

Hiusam avatar Mar 30 '23 02:03 Hiusam