discoelysiumLW comments

Repositories
Issues
Comments

Results 3 comments of


                                            discoelysiumLW

ValueError: weight is on the meta device, we need a `value` to put in on cpu.

I ran into this error " ValueError: weight is on the meta device, we need a `value` to put in on cpu." while loading llama-7B in 8 bits as well....

RuntimeError: module must have its parameters and buffers on device cuda:0 (device_ids[0]) but found one of them on device: cuda:1 0%|

If you load the pre-trained model sequentially seems to fix the problem: `model = transformers.AutoModelForCausalLM.from_pretrained( './llama-7b-hf', load_in_8bit=True, device_map='sequential', )`

AttributeError: module transformers.models.llama has no attribute LLaMATokenizer

For right now the model should be load as: AutoModelForCausalLM also the tokenizer: AutoTokenizer