jidandan666

Results 8 comments of jidandan666

RuntimeError: "triu_tril_cuda_template" not implemented for 'BFloat16'

你是改了代码，用半精度加载的吗

用了8块a100-40g 运行llama3-70b-instruct 提示如下错误

运行命令怎么写的

12G显卡跑不起来，可以把模型加载在两张卡上吗

> 我是这么改的：在main的参数加上model_parallel_size:int=2,generator = Llama.build加上model_parallel_size=model_parallel_size, 你下载的是哪个模型，.pth的还是safetensors。是不是如果用原始的一个.pth模型，不能分到两张显卡上去？

Trying to set a tensor of shape torch.Size([1024, 4096]) in "weight" (which has shape torch.Size([4096, 4096])), this look incorrect.

> llama3 8B 即使用了GQA，然而在llama2中7B和13B是没有使用GQA的，只有70B才使用了GQA，注意修改默认的n_kv_heads和n_heads即可。 n_kv_heads和n_heads改成什么，能告知一下吗。谢谢了

Trying to set a tensor of shape torch.Size([1024, 4096]) in "weight" (which has shape torch.Size([4096, 4096])), this look incorrect.

"num_hidden_layers": 32, "num_key_value_heads": 8, 是不是把config.py里面的这两个改成一样就行了

Trying to set a tensor of shape torch.Size([1024, 4096]) in "weight" (which has shape torch.Size([4096, 4096])), this look incorrect.

> problem has been successfully solved by updating transformers to 4.40.1, the lastest version that seems to support llama3. Yes you hear me! Though as easy and stupid as it...

Trying to set a tensor of shape torch.Size([1024, 4096]) in "weight" (which has shape torch.Size([4096, 4096])), this look incorrect.

> bitsandbytes > > > problem has been successfully solved by updating transformers to 4.40.1, the lastest version that seems to support llama3. Yes you hear me! Though as easy...

[BUG] 文件对话功能报错，上传文件进行文件对话，报错

同问