zyf-gh
zyf-gh
Thank you! It is indeed caused by the null pointer.
I don't quite understand. I got the model_res_acc.json of the exported model. The following is part of the file. It contains multiple thresholds. How can I get `shadowLayers` based on...
Does this method also work for Mixture-of-Experts(MoE) models?
If I quantize a MoE model and implement the definition, configuration, and tokenizer of the MoE model, I still cannot use the NPU of mllm to accelerate the MoE model,...
> Our KVCache is different from PyTorch's KV Cache Layout. Since you need to transfer the data to another device, you can reorganize the memory layout of our KVCache before...