lightllm icon indicating copy to clipboard operation
lightllm copied to clipboard
verified publisher icon ModelTC

An error occurred while deploying the 4bit version of Yi-34B-Chat

Open wx971025 opened this issue 2 years ago • 1 comments

AttributeError: 'LlamaTransformerLayerWeight' object has no attribute 'q_weight_' The 4bit version seems to be only "att_norm_weight_" and "ffn_norm_weight". When will the 4bit version be supported?

wx971025 avatar Nov 29 '23 03:11 wx971025

@wx971025 we will support this soon. Now we are supporting some feature for router and splitfuse.

hiworldwzj avatar Nov 29 '23 03:11 hiworldwzj