llama.cpp icon indicating copy to clipboard operation
llama.cpp copied to clipboard
Published 1 month ago • verified publisher icon ggml-org

[CANN]Support OP MUL_MAT_ID

Open noemotiovon opened this issue 8 months ago • 0 comments

Why is this PR needed?

Add support for the MUL_MAT_ID operator required by MOE models.

Op Test

Backend 1/2: CANN0
ggml_backend_cann_context: device 0 async operator submission is OFF
  Device description: Ascend910B3
  Device memory: 62432 MB (62145 MB free)

  MUL_MAT_ID(type_a=F32,type_b=F32,n_mats=4,n_used=1,b=0,m=512,n=1,k=256): new_pool_for_device: device 0 use vmm pool
OK
  MUL_MAT_ID(type_a=F32,type_b=F32,n_mats=4,n_used=1,b=0,m=512,n=32,k=256): OK
  MUL_MAT_ID(type_a=F32,type_b=F32,n_mats=4,n_used=1,b=0,m=512,n=129,k=256): OK
  MUL_MAT_ID(type_a=F32,type_b=F32,n_mats=4,n_used=1,b=1,m=512,n=1,k=256): OK
  MUL_MAT_ID(type_a=F32,type_b=F32,n_mats=4,n_used=1,b=1,m=512,n=32,k=256): OK
  MUL_MAT_ID(type_a=F32,type_b=F32,n_mats=4,n_used=1,b=1,m=512,n=129,k=256): OK
  MUL_MAT_ID(type_a=F32,type_b=F32,n_mats=4,n_used=2,b=0,m=512,n=1,k=256): OK
  MUL_MAT_ID(type_a=F32,type_b=F32,n_mats=4,n_used=2,b=0,m=512,n=32,k=256): OK
  MUL_MAT_ID(type_a=F32,type_b=F32,n_mats=4,n_used=2,b=0,m=512,n=129,k=256): OK
  MUL_MAT_ID(type_a=F32,type_b=F32,n_mats=4,n_used=2,b=1,m=512,n=1,k=256): OK
  MUL_MAT_ID(type_a=F32,type_b=F32,n_mats=4,n_used=2,b=1,m=512,n=32,k=256): OK
  MUL_MAT_ID(type_a=F32,type_b=F32,n_mats=4,n_used=2,b=1,m=512,n=129,k=256): OK
  MUL_MAT_ID(type_a=F32,type_b=F32,n_mats=4,n_used=4,b=0,m=512,n=1,k=256): OK
  MUL_MAT_ID(type_a=F32,type_b=F32,n_mats=4,n_used=4,b=0,m=512,n=32,k=256): OK
  MUL_MAT_ID(type_a=F32,type_b=F32,n_mats=4,n_used=4,b=0,m=512,n=129,k=256): OK
  MUL_MAT_ID(type_a=F32,type_b=F32,n_mats=4,n_used=4,b=1,m=512,n=1,k=256): OK
  MUL_MAT_ID(type_a=F32,type_b=F32,n_mats=4,n_used=4,b=1,m=512,n=32,k=256): OK
  MUL_MAT_ID(type_a=F32,type_b=F32,n_mats=4,n_used=4,b=1,m=512,n=129,k=256): OK
  MUL_MAT_ID(type_a=F32,type_b=F32,n_mats=8,n_used=1,b=0,m=512,n=1,k=256): OK
  MUL_MAT_ID(type_a=F32,type_b=F32,n_mats=8,n_used=1,b=0,m=512,n=32,k=256): OK
  MUL_MAT_ID(type_a=F32,type_b=F32,n_mats=8,n_used=1,b=0,m=512,n=129,k=256): OK
  MUL_MAT_ID(type_a=F32,type_b=F32,n_mats=8,n_used=1,b=1,m=512,n=1,k=256): OK
  MUL_MAT_ID(type_a=F32,type_b=F32,n_mats=8,n_used=1,b=1,m=512,n=32,k=256): OK
  MUL_MAT_ID(type_a=F32,type_b=F32,n_mats=8,n_used=1,b=1,m=512,n=129,k=256): OK
  MUL_MAT_ID(type_a=F32,type_b=F32,n_mats=8,n_used=2,b=0,m=512,n=1,k=256): OK
  MUL_MAT_ID(type_a=F32,type_b=F32,n_mats=8,n_used=2,b=0,m=512,n=32,k=256): OK
  MUL_MAT_ID(type_a=F32,type_b=F32,n_mats=8,n_used=2,b=0,m=512,n=129,k=256): OK
  MUL_MAT_ID(type_a=F32,type_b=F32,n_mats=8,n_used=2,b=1,m=512,n=1,k=256): OK
  MUL_MAT_ID(type_a=F32,type_b=F32,n_mats=8,n_used=2,b=1,m=512,n=32,k=256): OK
  MUL_MAT_ID(type_a=F32,type_b=F32,n_mats=8,n_used=2,b=1,m=512,n=129,k=256): OK
  MUL_MAT_ID(type_a=F32,type_b=F32,n_mats=8,n_used=4,b=0,m=512,n=1,k=256): OK
  MUL_MAT_ID(type_a=F32,type_b=F32,n_mats=8,n_used=4,b=0,m=512,n=32,k=256): OK
  MUL_MAT_ID(type_a=F32,type_b=F32,n_mats=8,n_used=4,b=0,m=512,n=129,k=256): OK
  MUL_MAT_ID(type_a=F32,type_b=F32,n_mats=8,n_used=4,b=1,m=512,n=1,k=256): OK
  MUL_MAT_ID(type_a=F32,type_b=F32,n_mats=8,n_used=4,b=1,m=512,n=32,k=256): OK
  MUL_MAT_ID(type_a=F32,type_b=F32,n_mats=8,n_used=4,b=1,m=512,n=129,k=256): OK
  MUL_MAT_ID(type_a=F16,type_b=F32,n_mats=4,n_used=1,b=0,m=512,n=1,k=256): OK
  MUL_MAT_ID(type_a=F16,type_b=F32,n_mats=4,n_used=1,b=0,m=512,n=32,k=256): OK
  MUL_MAT_ID(type_a=F16,type_b=F32,n_mats=4,n_used=1,b=0,m=512,n=129,k=256): OK
  MUL_MAT_ID(type_a=F16,type_b=F32,n_mats=4,n_used=1,b=1,m=512,n=1,k=256): OK
  MUL_MAT_ID(type_a=F16,type_b=F32,n_mats=4,n_used=1,b=1,m=512,n=32,k=256): OK
  MUL_MAT_ID(type_a=F16,type_b=F32,n_mats=4,n_used=1,b=1,m=512,n=129,k=256): OK
  MUL_MAT_ID(type_a=F16,type_b=F32,n_mats=4,n_used=2,b=0,m=512,n=1,k=256): OK
  MUL_MAT_ID(type_a=F16,type_b=F32,n_mats=4,n_used=2,b=0,m=512,n=32,k=256): OK
  MUL_MAT_ID(type_a=F16,type_b=F32,n_mats=4,n_used=2,b=0,m=512,n=129,k=256): OK
  MUL_MAT_ID(type_a=F16,type_b=F32,n_mats=4,n_used=2,b=1,m=512,n=1,k=256): OK
  MUL_MAT_ID(type_a=F16,type_b=F32,n_mats=4,n_used=2,b=1,m=512,n=32,k=256): OK
  MUL_MAT_ID(type_a=F16,type_b=F32,n_mats=4,n_used=2,b=1,m=512,n=129,k=256): OK
  MUL_MAT_ID(type_a=F16,type_b=F32,n_mats=4,n_used=4,b=0,m=512,n=1,k=256): OK
  MUL_MAT_ID(type_a=F16,type_b=F32,n_mats=4,n_used=4,b=0,m=512,n=32,k=256): OK
  MUL_MAT_ID(type_a=F16,type_b=F32,n_mats=4,n_used=4,b=0,m=512,n=129,k=256): OK
  MUL_MAT_ID(type_a=F16,type_b=F32,n_mats=4,n_used=4,b=1,m=512,n=1,k=256): OK
  MUL_MAT_ID(type_a=F16,type_b=F32,n_mats=4,n_used=4,b=1,m=512,n=32,k=256): OK
  MUL_MAT_ID(type_a=F16,type_b=F32,n_mats=4,n_used=4,b=1,m=512,n=129,k=256): OK
  MUL_MAT_ID(type_a=F16,type_b=F32,n_mats=8,n_used=1,b=0,m=512,n=1,k=256): OK
  MUL_MAT_ID(type_a=F16,type_b=F32,n_mats=8,n_used=1,b=0,m=512,n=32,k=256): OK
  MUL_MAT_ID(type_a=F16,type_b=F32,n_mats=8,n_used=1,b=0,m=512,n=129,k=256): OK
  MUL_MAT_ID(type_a=F16,type_b=F32,n_mats=8,n_used=1,b=1,m=512,n=1,k=256): OK
  MUL_MAT_ID(type_a=F16,type_b=F32,n_mats=8,n_used=1,b=1,m=512,n=32,k=256): OK
  MUL_MAT_ID(type_a=F16,type_b=F32,n_mats=8,n_used=1,b=1,m=512,n=129,k=256): OK
  MUL_MAT_ID(type_a=F16,type_b=F32,n_mats=8,n_used=2,b=0,m=512,n=1,k=256): OK
  MUL_MAT_ID(type_a=F16,type_b=F32,n_mats=8,n_used=2,b=0,m=512,n=32,k=256): OK
  MUL_MAT_ID(type_a=F16,type_b=F32,n_mats=8,n_used=2,b=0,m=512,n=129,k=256): OK
  MUL_MAT_ID(type_a=F16,type_b=F32,n_mats=8,n_used=2,b=1,m=512,n=1,k=256): OK
  MUL_MAT_ID(type_a=F16,type_b=F32,n_mats=8,n_used=2,b=1,m=512,n=32,k=256): OK
  MUL_MAT_ID(type_a=F16,type_b=F32,n_mats=8,n_used=2,b=1,m=512,n=129,k=256): OK
  MUL_MAT_ID(type_a=F16,type_b=F32,n_mats=8,n_used=4,b=0,m=512,n=1,k=256): OK
  MUL_MAT_ID(type_a=F16,type_b=F32,n_mats=8,n_used=4,b=0,m=512,n=32,k=256): OK
  MUL_MAT_ID(type_a=F16,type_b=F32,n_mats=8,n_used=4,b=0,m=512,n=129,k=256): OK
  MUL_MAT_ID(type_a=F16,type_b=F32,n_mats=8,n_used=4,b=1,m=512,n=1,k=256): OK
  MUL_MAT_ID(type_a=F16,type_b=F32,n_mats=8,n_used=4,b=1,m=512,n=32,k=256): OK
  MUL_MAT_ID(type_a=F16,type_b=F32,n_mats=8,n_used=4,b=1,m=512,n=129,k=256): OK
  MUL_MAT_ID(type_a=Q8_0,type_b=F32,n_mats=4,n_used=1,b=0,m=512,n=1,k=256): not supported [CANN0] 
  MUL_MAT_ID(type_a=Q8_0,type_b=F32,n_mats=4,n_used=1,b=0,m=512,n=32,k=256): not supported [CANN0] 
  MUL_MAT_ID(type_a=Q8_0,type_b=F32,n_mats=4,n_used=1,b=0,m=512,n=129,k=256): not supported [CANN0] 
  MUL_MAT_ID(type_a=Q8_0,type_b=F32,n_mats=4,n_used=1,b=1,m=512,n=1,k=256): not supported [CANN0] 
  MUL_MAT_ID(type_a=Q8_0,type_b=F32,n_mats=4,n_used=1,b=1,m=512,n=32,k=256): not supported [CANN0] 
  MUL_MAT_ID(type_a=Q8_0,type_b=F32,n_mats=4,n_used=1,b=1,m=512,n=129,k=256): not supported [CANN0] 
  MUL_MAT_ID(type_a=Q8_0,type_b=F32,n_mats=4,n_used=2,b=0,m=512,n=1,k=256): not supported [CANN0] 
  MUL_MAT_ID(type_a=Q8_0,type_b=F32,n_mats=4,n_used=2,b=0,m=512,n=32,k=256): not supported [CANN0] 
  MUL_MAT_ID(type_a=Q8_0,type_b=F32,n_mats=4,n_used=2,b=0,m=512,n=129,k=256): not supported [CANN0] 
  MUL_MAT_ID(type_a=Q8_0,type_b=F32,n_mats=4,n_used=2,b=1,m=512,n=1,k=256): not supported [CANN0] 
  MUL_MAT_ID(type_a=Q8_0,type_b=F32,n_mats=4,n_used=2,b=1,m=512,n=32,k=256): not supported [CANN0] 
  MUL_MAT_ID(type_a=Q8_0,type_b=F32,n_mats=4,n_used=2,b=1,m=512,n=129,k=256): not supported [CANN0] 
  MUL_MAT_ID(type_a=Q8_0,type_b=F32,n_mats=4,n_used=4,b=0,m=512,n=1,k=256): not supported [CANN0] 
  MUL_MAT_ID(type_a=Q8_0,type_b=F32,n_mats=4,n_used=4,b=0,m=512,n=32,k=256): not supported [CANN0] 
  MUL_MAT_ID(type_a=Q8_0,type_b=F32,n_mats=4,n_used=4,b=0,m=512,n=129,k=256): not supported [CANN0] 
  MUL_MAT_ID(type_a=Q8_0,type_b=F32,n_mats=4,n_used=4,b=1,m=512,n=1,k=256): not supported [CANN0] 
  MUL_MAT_ID(type_a=Q8_0,type_b=F32,n_mats=4,n_used=4,b=1,m=512,n=32,k=256): not supported [CANN0] 
  MUL_MAT_ID(type_a=Q8_0,type_b=F32,n_mats=4,n_used=4,b=1,m=512,n=129,k=256): not supported [CANN0] 
  MUL_MAT_ID(type_a=Q8_0,type_b=F32,n_mats=8,n_used=1,b=0,m=512,n=1,k=256): not supported [CANN0] 
  MUL_MAT_ID(type_a=Q8_0,type_b=F32,n_mats=8,n_used=1,b=0,m=512,n=32,k=256): not supported [CANN0] 
  MUL_MAT_ID(type_a=Q8_0,type_b=F32,n_mats=8,n_used=1,b=0,m=512,n=129,k=256): not supported [CANN0] 
  MUL_MAT_ID(type_a=Q8_0,type_b=F32,n_mats=8,n_used=1,b=1,m=512,n=1,k=256): not supported [CANN0] 
  MUL_MAT_ID(type_a=Q8_0,type_b=F32,n_mats=8,n_used=1,b=1,m=512,n=32,k=256): not supported [CANN0] 
  MUL_MAT_ID(type_a=Q8_0,type_b=F32,n_mats=8,n_used=1,b=1,m=512,n=129,k=256): not supported [CANN0] 
  MUL_MAT_ID(type_a=Q8_0,type_b=F32,n_mats=8,n_used=2,b=0,m=512,n=1,k=256): not supported [CANN0] 
  MUL_MAT_ID(type_a=Q8_0,type_b=F32,n_mats=8,n_used=2,b=0,m=512,n=32,k=256): not supported [CANN0] 
  MUL_MAT_ID(type_a=Q8_0,type_b=F32,n_mats=8,n_used=2,b=0,m=512,n=129,k=256): not supported [CANN0] 
  MUL_MAT_ID(type_a=Q8_0,type_b=F32,n_mats=8,n_used=2,b=1,m=512,n=1,k=256): not supported [CANN0] 
  MUL_MAT_ID(type_a=Q8_0,type_b=F32,n_mats=8,n_used=2,b=1,m=512,n=32,k=256): not supported [CANN0] 
  MUL_MAT_ID(type_a=Q8_0,type_b=F32,n_mats=8,n_used=2,b=1,m=512,n=129,k=256): not supported [CANN0] 
  MUL_MAT_ID(type_a=Q8_0,type_b=F32,n_mats=8,n_used=4,b=0,m=512,n=1,k=256): not supported [CANN0] 
  MUL_MAT_ID(type_a=Q8_0,type_b=F32,n_mats=8,n_used=4,b=0,m=512,n=32,k=256): not supported [CANN0] 
  MUL_MAT_ID(type_a=Q8_0,type_b=F32,n_mats=8,n_used=4,b=0,m=512,n=129,k=256): not supported [CANN0] 
  MUL_MAT_ID(type_a=Q8_0,type_b=F32,n_mats=8,n_used=4,b=1,m=512,n=1,k=256): not supported [CANN0] 
  MUL_MAT_ID(type_a=Q8_0,type_b=F32,n_mats=8,n_used=4,b=1,m=512,n=32,k=256): not supported [CANN0] 
  MUL_MAT_ID(type_a=Q8_0,type_b=F32,n_mats=8,n_used=4,b=1,m=512,n=129,k=256): not supported [CANN0] 
  MUL_MAT_ID(type_a=Q4_0,type_b=F32,n_mats=4,n_used=1,b=0,m=512,n=1,k=256): not supported [CANN0] 
  MUL_MAT_ID(type_a=Q4_0,type_b=F32,n_mats=4,n_used=1,b=0,m=512,n=32,k=256): not supported [CANN0] 
  MUL_MAT_ID(type_a=Q4_0,type_b=F32,n_mats=4,n_used=1,b=0,m=512,n=129,k=256): not supported [CANN0] 
  MUL_MAT_ID(type_a=Q4_0,type_b=F32,n_mats=4,n_used=1,b=1,m=512,n=1,k=256): not supported [CANN0] 
  MUL_MAT_ID(type_a=Q4_0,type_b=F32,n_mats=4,n_used=1,b=1,m=512,n=32,k=256): not supported [CANN0] 
  MUL_MAT_ID(type_a=Q4_0,type_b=F32,n_mats=4,n_used=1,b=1,m=512,n=129,k=256): not supported [CANN0] 
  MUL_MAT_ID(type_a=Q4_0,type_b=F32,n_mats=4,n_used=2,b=0,m=512,n=1,k=256): not supported [CANN0] 
  MUL_MAT_ID(type_a=Q4_0,type_b=F32,n_mats=4,n_used=2,b=0,m=512,n=32,k=256): not supported [CANN0] 
  MUL_MAT_ID(type_a=Q4_0,type_b=F32,n_mats=4,n_used=2,b=0,m=512,n=129,k=256): not supported [CANN0] 
  MUL_MAT_ID(type_a=Q4_0,type_b=F32,n_mats=4,n_used=2,b=1,m=512,n=1,k=256): not supported [CANN0] 
  MUL_MAT_ID(type_a=Q4_0,type_b=F32,n_mats=4,n_used=2,b=1,m=512,n=32,k=256): not supported [CANN0] 
  MUL_MAT_ID(type_a=Q4_0,type_b=F32,n_mats=4,n_used=2,b=1,m=512,n=129,k=256): not supported [CANN0] 
  MUL_MAT_ID(type_a=Q4_0,type_b=F32,n_mats=4,n_used=4,b=0,m=512,n=1,k=256): not supported [CANN0] 
  MUL_MAT_ID(type_a=Q4_0,type_b=F32,n_mats=4,n_used=4,b=0,m=512,n=32,k=256): not supported [CANN0] 
  MUL_MAT_ID(type_a=Q4_0,type_b=F32,n_mats=4,n_used=4,b=0,m=512,n=129,k=256): not supported [CANN0] 
  MUL_MAT_ID(type_a=Q4_0,type_b=F32,n_mats=4,n_used=4,b=1,m=512,n=1,k=256): not supported [CANN0] 
  MUL_MAT_ID(type_a=Q4_0,type_b=F32,n_mats=4,n_used=4,b=1,m=512,n=32,k=256): not supported [CANN0] 
  MUL_MAT_ID(type_a=Q4_0,type_b=F32,n_mats=4,n_used=4,b=1,m=512,n=129,k=256): not supported [CANN0] 
  MUL_MAT_ID(type_a=Q4_0,type_b=F32,n_mats=8,n_used=1,b=0,m=512,n=1,k=256): not supported [CANN0] 
  MUL_MAT_ID(type_a=Q4_0,type_b=F32,n_mats=8,n_used=1,b=0,m=512,n=32,k=256): not supported [CANN0] 
  MUL_MAT_ID(type_a=Q4_0,type_b=F32,n_mats=8,n_used=1,b=0,m=512,n=129,k=256): not supported [CANN0] 
  MUL_MAT_ID(type_a=Q4_0,type_b=F32,n_mats=8,n_used=1,b=1,m=512,n=1,k=256): not supported [CANN0] 
  MUL_MAT_ID(type_a=Q4_0,type_b=F32,n_mats=8,n_used=1,b=1,m=512,n=32,k=256): not supported [CANN0] 
  MUL_MAT_ID(type_a=Q4_0,type_b=F32,n_mats=8,n_used=1,b=1,m=512,n=129,k=256): not supported [CANN0] 
  MUL_MAT_ID(type_a=Q4_0,type_b=F32,n_mats=8,n_used=2,b=0,m=512,n=1,k=256): not supported [CANN0] 
  MUL_MAT_ID(type_a=Q4_0,type_b=F32,n_mats=8,n_used=2,b=0,m=512,n=32,k=256): not supported [CANN0] 
  MUL_MAT_ID(type_a=Q4_0,type_b=F32,n_mats=8,n_used=2,b=0,m=512,n=129,k=256): not supported [CANN0] 
  MUL_MAT_ID(type_a=Q4_0,type_b=F32,n_mats=8,n_used=2,b=1,m=512,n=1,k=256): not supported [CANN0] 
  MUL_MAT_ID(type_a=Q4_0,type_b=F32,n_mats=8,n_used=2,b=1,m=512,n=32,k=256): not supported [CANN0] 
  MUL_MAT_ID(type_a=Q4_0,type_b=F32,n_mats=8,n_used=2,b=1,m=512,n=129,k=256): not supported [CANN0] 
  MUL_MAT_ID(type_a=Q4_0,type_b=F32,n_mats=8,n_used=4,b=0,m=512,n=1,k=256): not supported [CANN0] 
  MUL_MAT_ID(type_a=Q4_0,type_b=F32,n_mats=8,n_used=4,b=0,m=512,n=32,k=256): not supported [CANN0] 
  MUL_MAT_ID(type_a=Q4_0,type_b=F32,n_mats=8,n_used=4,b=0,m=512,n=129,k=256): not supported [CANN0] 
  MUL_MAT_ID(type_a=Q4_0,type_b=F32,n_mats=8,n_used=4,b=1,m=512,n=1,k=256): not supported [CANN0] 
  MUL_MAT_ID(type_a=Q4_0,type_b=F32,n_mats=8,n_used=4,b=1,m=512,n=32,k=256): not supported [CANN0] 
  MUL_MAT_ID(type_a=Q4_0,type_b=F32,n_mats=8,n_used=4,b=1,m=512,n=129,k=256): not supported [CANN0] 
  MUL_MAT_ID(type_a=Q4_1,type_b=F32,n_mats=4,n_used=1,b=0,m=512,n=1,k=256): not supported [CANN0] 
  MUL_MAT_ID(type_a=Q4_1,type_b=F32,n_mats=4,n_used=1,b=0,m=512,n=32,k=256): not supported [CANN0] 
  MUL_MAT_ID(type_a=Q4_1,type_b=F32,n_mats=4,n_used=1,b=0,m=512,n=129,k=256): not supported [CANN0] 
  MUL_MAT_ID(type_a=Q4_1,type_b=F32,n_mats=4,n_used=1,b=1,m=512,n=1,k=256): not supported [CANN0] 
  MUL_MAT_ID(type_a=Q4_1,type_b=F32,n_mats=4,n_used=1,b=1,m=512,n=32,k=256): not supported [CANN0] 
  MUL_MAT_ID(type_a=Q4_1,type_b=F32,n_mats=4,n_used=1,b=1,m=512,n=129,k=256): not supported [CANN0] 
  MUL_MAT_ID(type_a=Q4_1,type_b=F32,n_mats=4,n_used=2,b=0,m=512,n=1,k=256): not supported [CANN0] 
  MUL_MAT_ID(type_a=Q4_1,type_b=F32,n_mats=4,n_used=2,b=0,m=512,n=32,k=256): not supported [CANN0] 
  MUL_MAT_ID(type_a=Q4_1,type_b=F32,n_mats=4,n_used=2,b=0,m=512,n=129,k=256): not supported [CANN0] 
  MUL_MAT_ID(type_a=Q4_1,type_b=F32,n_mats=4,n_used=2,b=1,m=512,n=1,k=256): not supported [CANN0] 
  MUL_MAT_ID(type_a=Q4_1,type_b=F32,n_mats=4,n_used=2,b=1,m=512,n=32,k=256): not supported [CANN0] 
  MUL_MAT_ID(type_a=Q4_1,type_b=F32,n_mats=4,n_used=2,b=1,m=512,n=129,k=256): not supported [CANN0] 
  MUL_MAT_ID(type_a=Q4_1,type_b=F32,n_mats=4,n_used=4,b=0,m=512,n=1,k=256): not supported [CANN0] 
  MUL_MAT_ID(type_a=Q4_1,type_b=F32,n_mats=4,n_used=4,b=0,m=512,n=32,k=256): not supported [CANN0] 
  MUL_MAT_ID(type_a=Q4_1,type_b=F32,n_mats=4,n_used=4,b=0,m=512,n=129,k=256): not supported [CANN0] 
  MUL_MAT_ID(type_a=Q4_1,type_b=F32,n_mats=4,n_used=4,b=1,m=512,n=1,k=256): not supported [CANN0] 
  MUL_MAT_ID(type_a=Q4_1,type_b=F32,n_mats=4,n_used=4,b=1,m=512,n=32,k=256): not supported [CANN0] 
  MUL_MAT_ID(type_a=Q4_1,type_b=F32,n_mats=4,n_used=4,b=1,m=512,n=129,k=256): not supported [CANN0] 
  MUL_MAT_ID(type_a=Q4_1,type_b=F32,n_mats=8,n_used=1,b=0,m=512,n=1,k=256): not supported [CANN0] 
  MUL_MAT_ID(type_a=Q4_1,type_b=F32,n_mats=8,n_used=1,b=0,m=512,n=32,k=256): not supported [CANN0] 
  MUL_MAT_ID(type_a=Q4_1,type_b=F32,n_mats=8,n_used=1,b=0,m=512,n=129,k=256): not supported [CANN0] 
  MUL_MAT_ID(type_a=Q4_1,type_b=F32,n_mats=8,n_used=1,b=1,m=512,n=1,k=256): not supported [CANN0] 
  MUL_MAT_ID(type_a=Q4_1,type_b=F32,n_mats=8,n_used=1,b=1,m=512,n=32,k=256): not supported [CANN0] 
  MUL_MAT_ID(type_a=Q4_1,type_b=F32,n_mats=8,n_used=1,b=1,m=512,n=129,k=256): not supported [CANN0] 
  MUL_MAT_ID(type_a=Q4_1,type_b=F32,n_mats=8,n_used=2,b=0,m=512,n=1,k=256): not supported [CANN0] 
  MUL_MAT_ID(type_a=Q4_1,type_b=F32,n_mats=8,n_used=2,b=0,m=512,n=32,k=256): not supported [CANN0] 
  MUL_MAT_ID(type_a=Q4_1,type_b=F32,n_mats=8,n_used=2,b=0,m=512,n=129,k=256): not supported [CANN0] 
  MUL_MAT_ID(type_a=Q4_1,type_b=F32,n_mats=8,n_used=2,b=1,m=512,n=1,k=256): not supported [CANN0] 
  MUL_MAT_ID(type_a=Q4_1,type_b=F32,n_mats=8,n_used=2,b=1,m=512,n=32,k=256): not supported [CANN0] 
  MUL_MAT_ID(type_a=Q4_1,type_b=F32,n_mats=8,n_used=2,b=1,m=512,n=129,k=256): not supported [CANN0] 
  MUL_MAT_ID(type_a=Q4_1,type_b=F32,n_mats=8,n_used=4,b=0,m=512,n=1,k=256): not supported [CANN0] 
  MUL_MAT_ID(type_a=Q4_1,type_b=F32,n_mats=8,n_used=4,b=0,m=512,n=32,k=256): not supported [CANN0] 
  MUL_MAT_ID(type_a=Q4_1,type_b=F32,n_mats=8,n_used=4,b=0,m=512,n=129,k=256): not supported [CANN0] 
  MUL_MAT_ID(type_a=Q4_1,type_b=F32,n_mats=8,n_used=4,b=1,m=512,n=1,k=256): not supported [CANN0] 
  MUL_MAT_ID(type_a=Q4_1,type_b=F32,n_mats=8,n_used=4,b=1,m=512,n=32,k=256): not supported [CANN0] 
  MUL_MAT_ID(type_a=Q4_1,type_b=F32,n_mats=8,n_used=4,b=1,m=512,n=129,k=256): not supported [CANN0] 
  MUL_MAT_ID(type_a=UNKNOWN,type_b=F32,n_mats=4,n_used=1,b=0,m=512,n=1,k=256): not supported [CANN0] 
  MUL_MAT_ID(type_a=UNKNOWN,type_b=F32,n_mats=4,n_used=1,b=0,m=512,n=32,k=256): not supported [CANN0] 
  MUL_MAT_ID(type_a=UNKNOWN,type_b=F32,n_mats=4,n_used=1,b=0,m=512,n=129,k=256): not supported [CANN0] 
  MUL_MAT_ID(type_a=UNKNOWN,type_b=F32,n_mats=4,n_used=1,b=1,m=512,n=1,k=256): not supported [CANN0] 
  MUL_MAT_ID(type_a=UNKNOWN,type_b=F32,n_mats=4,n_used=1,b=1,m=512,n=32,k=256): not supported [CANN0] 
  MUL_MAT_ID(type_a=UNKNOWN,type_b=F32,n_mats=4,n_used=1,b=1,m=512,n=129,k=256): not supported [CANN0] 
  MUL_MAT_ID(type_a=UNKNOWN,type_b=F32,n_mats=4,n_used=2,b=0,m=512,n=1,k=256): not supported [CANN0] 
  MUL_MAT_ID(type_a=UNKNOWN,type_b=F32,n_mats=4,n_used=2,b=0,m=512,n=32,k=256): not supported [CANN0] 
  MUL_MAT_ID(type_a=UNKNOWN,type_b=F32,n_mats=4,n_used=2,b=0,m=512,n=129,k=256): not supported [CANN0] 
  MUL_MAT_ID(type_a=UNKNOWN,type_b=F32,n_mats=4,n_used=2,b=1,m=512,n=1,k=256): not supported [CANN0] 
  MUL_MAT_ID(type_a=UNKNOWN,type_b=F32,n_mats=4,n_used=2,b=1,m=512,n=32,k=256): not supported [CANN0] 
  MUL_MAT_ID(type_a=UNKNOWN,type_b=F32,n_mats=4,n_used=2,b=1,m=512,n=129,k=256): not supported [CANN0] 
  MUL_MAT_ID(type_a=UNKNOWN,type_b=F32,n_mats=4,n_used=4,b=0,m=512,n=1,k=256): not supported [CANN0] 
  MUL_MAT_ID(type_a=UNKNOWN,type_b=F32,n_mats=4,n_used=4,b=0,m=512,n=32,k=256): not supported [CANN0] 
  MUL_MAT_ID(type_a=UNKNOWN,type_b=F32,n_mats=4,n_used=4,b=0,m=512,n=129,k=256): not supported [CANN0] 
  MUL_MAT_ID(type_a=UNKNOWN,type_b=F32,n_mats=4,n_used=4,b=1,m=512,n=1,k=256): not supported [CANN0] 
  MUL_MAT_ID(type_a=UNKNOWN,type_b=F32,n_mats=4,n_used=4,b=1,m=512,n=32,k=256): not supported [CANN0] 
  MUL_MAT_ID(type_a=UNKNOWN,type_b=F32,n_mats=4,n_used=4,b=1,m=512,n=129,k=256): not supported [CANN0] 
  MUL_MAT_ID(type_a=UNKNOWN,type_b=F32,n_mats=8,n_used=1,b=0,m=512,n=1,k=256): not supported [CANN0] 
  MUL_MAT_ID(type_a=UNKNOWN,type_b=F32,n_mats=8,n_used=1,b=0,m=512,n=32,k=256): not supported [CANN0] 
  MUL_MAT_ID(type_a=UNKNOWN,type_b=F32,n_mats=8,n_used=1,b=0,m=512,n=129,k=256): not supported [CANN0] 
  MUL_MAT_ID(type_a=UNKNOWN,type_b=F32,n_mats=8,n_used=1,b=1,m=512,n=1,k=256): not supported [CANN0] 
  MUL_MAT_ID(type_a=UNKNOWN,type_b=F32,n_mats=8,n_used=1,b=1,m=512,n=32,k=256): not supported [CANN0] 
  MUL_MAT_ID(type_a=UNKNOWN,type_b=F32,n_mats=8,n_used=1,b=1,m=512,n=129,k=256): not supported [CANN0] 
  MUL_MAT_ID(type_a=UNKNOWN,type_b=F32,n_mats=8,n_used=2,b=0,m=512,n=1,k=256): not supported [CANN0] 
  MUL_MAT_ID(type_a=UNKNOWN,type_b=F32,n_mats=8,n_used=2,b=0,m=512,n=32,k=256): not supported [CANN0] 
  MUL_MAT_ID(type_a=UNKNOWN,type_b=F32,n_mats=8,n_used=2,b=0,m=512,n=129,k=256): not supported [CANN0] 
  MUL_MAT_ID(type_a=UNKNOWN,type_b=F32,n_mats=8,n_used=2,b=1,m=512,n=1,k=256): not supported [CANN0] 
  MUL_MAT_ID(type_a=UNKNOWN,type_b=F32,n_mats=8,n_used=2,b=1,m=512,n=32,k=256): not supported [CANN0] 
  MUL_MAT_ID(type_a=UNKNOWN,type_b=F32,n_mats=8,n_used=2,b=1,m=512,n=129,k=256): not supported [CANN0] 
  MUL_MAT_ID(type_a=UNKNOWN,type_b=F32,n_mats=8,n_used=4,b=0,m=512,n=1,k=256): not supported [CANN0] 
  MUL_MAT_ID(type_a=UNKNOWN,type_b=F32,n_mats=8,n_used=4,b=0,m=512,n=32,k=256): not supported [CANN0] 
  MUL_MAT_ID(type_a=UNKNOWN,type_b=F32,n_mats=8,n_used=4,b=0,m=512,n=129,k=256): not supported [CANN0] 
  MUL_MAT_ID(type_a=UNKNOWN,type_b=F32,n_mats=8,n_used=4,b=1,m=512,n=1,k=256): not supported [CANN0] 
  MUL_MAT_ID(type_a=UNKNOWN,type_b=F32,n_mats=8,n_used=4,b=1,m=512,n=32,k=256): not supported [CANN0] 
  MUL_MAT_ID(type_a=UNKNOWN,type_b=F32,n_mats=8,n_used=4,b=1,m=512,n=129,k=256): not supported [CANN0] 
  MUL_MAT_ID(type_a=UNKNOWN,type_b=F32,n_mats=4,n_used=1,b=0,m=512,n=1,k=256): not supported [CANN0] 
  MUL_MAT_ID(type_a=UNKNOWN,type_b=F32,n_mats=4,n_used=1,b=0,m=512,n=32,k=256): not supported [CANN0] 
  MUL_MAT_ID(type_a=UNKNOWN,type_b=F32,n_mats=4,n_used=1,b=0,m=512,n=129,k=256): not supported [CANN0] 
  MUL_MAT_ID(type_a=UNKNOWN,type_b=F32,n_mats=4,n_used=1,b=1,m=512,n=1,k=256): not supported [CANN0] 
  MUL_MAT_ID(type_a=UNKNOWN,type_b=F32,n_mats=4,n_used=1,b=1,m=512,n=32,k=256): not supported [CANN0] 
  MUL_MAT_ID(type_a=UNKNOWN,type_b=F32,n_mats=4,n_used=1,b=1,m=512,n=129,k=256): not supported [CANN0] 
  MUL_MAT_ID(type_a=UNKNOWN,type_b=F32,n_mats=4,n_used=2,b=0,m=512,n=1,k=256): not supported [CANN0] 
  MUL_MAT_ID(type_a=UNKNOWN,type_b=F32,n_mats=4,n_used=2,b=0,m=512,n=32,k=256): not supported [CANN0] 
  MUL_MAT_ID(type_a=UNKNOWN,type_b=F32,n_mats=4,n_used=2,b=0,m=512,n=129,k=256): not supported [CANN0] 
  MUL_MAT_ID(type_a=UNKNOWN,type_b=F32,n_mats=4,n_used=2,b=1,m=512,n=1,k=256): not supported [CANN0] 
  MUL_MAT_ID(type_a=UNKNOWN,type_b=F32,n_mats=4,n_used=2,b=1,m=512,n=32,k=256): not supported [CANN0] 
  MUL_MAT_ID(type_a=UNKNOWN,type_b=F32,n_mats=4,n_used=2,b=1,m=512,n=129,k=256): not supported [CANN0] 
  MUL_MAT_ID(type_a=UNKNOWN,type_b=F32,n_mats=4,n_used=4,b=0,m=512,n=1,k=256): not supported [CANN0] 
  MUL_MAT_ID(type_a=UNKNOWN,type_b=F32,n_mats=4,n_used=4,b=0,m=512,n=32,k=256): not supported [CANN0] 
  MUL_MAT_ID(type_a=UNKNOWN,type_b=F32,n_mats=4,n_used=4,b=0,m=512,n=129,k=256): not supported [CANN0] 
  MUL_MAT_ID(type_a=UNKNOWN,type_b=F32,n_mats=4,n_used=4,b=1,m=512,n=1,k=256): not supported [CANN0] 
  MUL_MAT_ID(type_a=UNKNOWN,type_b=F32,n_mats=4,n_used=4,b=1,m=512,n=32,k=256): not supported [CANN0] 
  MUL_MAT_ID(type_a=UNKNOWN,type_b=F32,n_mats=4,n_used=4,b=1,m=512,n=129,k=256): not supported [CANN0] 
  MUL_MAT_ID(type_a=UNKNOWN,type_b=F32,n_mats=8,n_used=1,b=0,m=512,n=1,k=256): not supported [CANN0] 
  MUL_MAT_ID(type_a=UNKNOWN,type_b=F32,n_mats=8,n_used=1,b=0,m=512,n=32,k=256): not supported [CANN0] 
  MUL_MAT_ID(type_a=UNKNOWN,type_b=F32,n_mats=8,n_used=1,b=0,m=512,n=129,k=256): not supported [CANN0] 
  MUL_MAT_ID(type_a=UNKNOWN,type_b=F32,n_mats=8,n_used=1,b=1,m=512,n=1,k=256): not supported [CANN0] 
  MUL_MAT_ID(type_a=UNKNOWN,type_b=F32,n_mats=8,n_used=1,b=1,m=512,n=32,k=256): not supported [CANN0] 
  MUL_MAT_ID(type_a=UNKNOWN,type_b=F32,n_mats=8,n_used=1,b=1,m=512,n=129,k=256): not supported [CANN0] 
  MUL_MAT_ID(type_a=UNKNOWN,type_b=F32,n_mats=8,n_used=2,b=0,m=512,n=1,k=256): not supported [CANN0] 
  MUL_MAT_ID(type_a=UNKNOWN,type_b=F32,n_mats=8,n_used=2,b=0,m=512,n=32,k=256): not supported [CANN0] 
  MUL_MAT_ID(type_a=UNKNOWN,type_b=F32,n_mats=8,n_used=2,b=0,m=512,n=129,k=256): not supported [CANN0] 
  MUL_MAT_ID(type_a=UNKNOWN,type_b=F32,n_mats=8,n_used=2,b=1,m=512,n=1,k=256): not supported [CANN0] 
  MUL_MAT_ID(type_a=UNKNOWN,type_b=F32,n_mats=8,n_used=2,b=1,m=512,n=32,k=256): not supported [CANN0] 
  MUL_MAT_ID(type_a=UNKNOWN,type_b=F32,n_mats=8,n_used=2,b=1,m=512,n=129,k=256): not supported [CANN0] 
  MUL_MAT_ID(type_a=UNKNOWN,type_b=F32,n_mats=8,n_used=4,b=0,m=512,n=1,k=256): not supported [CANN0] 
  MUL_MAT_ID(type_a=UNKNOWN,type_b=F32,n_mats=8,n_used=4,b=0,m=512,n=32,k=256): not supported [CANN0] 
  MUL_MAT_ID(type_a=UNKNOWN,type_b=F32,n_mats=8,n_used=4,b=0,m=512,n=129,k=256): not supported [CANN0] 
  MUL_MAT_ID(type_a=UNKNOWN,type_b=F32,n_mats=8,n_used=4,b=1,m=512,n=1,k=256): not supported [CANN0] 
  MUL_MAT_ID(type_a=UNKNOWN,type_b=F32,n_mats=8,n_used=4,b=1,m=512,n=32,k=256): not supported [CANN0] 
  MUL_MAT_ID(type_a=UNKNOWN,type_b=F32,n_mats=8,n_used=4,b=1,m=512,n=129,k=256): not supported [CANN0] 
  MUL_MAT_ID(type_a=Q4_1,type_b=F32,n_mats=4,n_used=2,b=0,m=512,n=1,k=256): not supported [CANN0] 
  MUL_MAT_ID(type_a=Q4_1,type_b=F32,n_mats=4,n_used=2,b=0,m=512,n=32,k=256): not supported [CANN0] 
  MUL_MAT_ID(type_a=Q5_0,type_b=F32,n_mats=4,n_used=2,b=0,m=512,n=1,k=256): not supported [CANN0] 
  MUL_MAT_ID(type_a=Q5_0,type_b=F32,n_mats=4,n_used=2,b=0,m=512,n=32,k=256): not supported [CANN0] 
  MUL_MAT_ID(type_a=Q5_1,type_b=F32,n_mats=4,n_used=2,b=0,m=512,n=1,k=256): not supported [CANN0] 
  MUL_MAT_ID(type_a=Q5_1,type_b=F32,n_mats=4,n_used=2,b=0,m=512,n=32,k=256): not supported [CANN0] 
  MUL_MAT_ID(type_a=Q8_0,type_b=F32,n_mats=4,n_used=2,b=0,m=512,n=1,k=256): not supported [CANN0] 
  MUL_MAT_ID(type_a=Q8_0,type_b=F32,n_mats=4,n_used=2,b=0,m=512,n=32,k=256): not supported [CANN0] 
  MUL_MAT_ID(type_a=UNKNOWN,type_b=F32,n_mats=4,n_used=2,b=0,m=512,n=1,k=256): not supported [CANN0] 
  MUL_MAT_ID(type_a=UNKNOWN,type_b=F32,n_mats=4,n_used=2,b=0,m=512,n=32,k=256): not supported [CANN0] 
  MUL_MAT_ID(type_a=UNKNOWN,type_b=F32,n_mats=4,n_used=2,b=0,m=512,n=1,k=256): not supported [CANN0] 
  MUL_MAT_ID(type_a=UNKNOWN,type_b=F32,n_mats=4,n_used=2,b=0,m=512,n=32,k=256): not supported [CANN0] 
  MUL_MAT_ID(type_a=UNKNOWN,type_b=F32,n_mats=4,n_used=2,b=0,m=512,n=1,k=256): not supported [CANN0] 
  MUL_MAT_ID(type_a=UNKNOWN,type_b=F32,n_mats=4,n_used=2,b=0,m=512,n=32,k=256): not supported [CANN0] 
  MUL_MAT_ID(type_a=UNKNOWN,type_b=F32,n_mats=4,n_used=2,b=0,m=512,n=1,k=256): not supported [CANN0] 
  MUL_MAT_ID(type_a=UNKNOWN,type_b=F32,n_mats=4,n_used=2,b=0,m=512,n=32,k=256): not supported [CANN0] 
  MUL_MAT_ID(type_a=UNKNOWN,type_b=F32,n_mats=4,n_used=2,b=0,m=512,n=1,k=256): not supported [CANN0] 
  MUL_MAT_ID(type_a=UNKNOWN,type_b=F32,n_mats=4,n_used=2,b=0,m=512,n=32,k=256): not supported [CANN0] 
  MUL_MAT_ID(type_a=UNKNOWN,type_b=F32,n_mats=4,n_used=2,b=0,m=512,n=1,k=256): not supported [CANN0] 
  MUL_MAT_ID(type_a=UNKNOWN,type_b=F32,n_mats=4,n_used=2,b=0,m=512,n=32,k=256): not supported [CANN0] 
  MUL_MAT_ID(type_a=UNKNOWN,type_b=F32,n_mats=4,n_used=2,b=0,m=512,n=1,k=256): not supported [CANN0] 
  MUL_MAT_ID(type_a=UNKNOWN,type_b=F32,n_mats=4,n_used=2,b=0,m=512,n=32,k=256): not supported [CANN0] 
  MUL_MAT_ID(type_a=UNKNOWN,type_b=F32,n_mats=4,n_used=2,b=0,m=512,n=1,k=256): not supported [CANN0] 
  MUL_MAT_ID(type_a=UNKNOWN,type_b=F32,n_mats=4,n_used=2,b=0,m=512,n=32,k=256): not supported [CANN0] 
  MUL_MAT_ID(type_a=UNKNOWN,type_b=F32,n_mats=4,n_used=2,b=0,m=512,n=1,k=256): not supported [CANN0] 
  MUL_MAT_ID(type_a=UNKNOWN,type_b=F32,n_mats=4,n_used=2,b=0,m=512,n=32,k=256): not supported [CANN0] 
  MUL_MAT_ID(type_a=UNKNOWN,type_b=F32,n_mats=4,n_used=2,b=0,m=512,n=1,k=256): not supported [CANN0] 
  MUL_MAT_ID(type_a=UNKNOWN,type_b=F32,n_mats=4,n_used=2,b=0,m=512,n=32,k=256): not supported [CANN0] 
  MUL_MAT_ID(type_a=UNKNOWN,type_b=F32,n_mats=4,n_used=2,b=0,m=512,n=1,k=256): not supported [CANN0] 
  MUL_MAT_ID(type_a=UNKNOWN,type_b=F32,n_mats=4,n_used=2,b=0,m=512,n=32,k=256): not supported [CANN0] 
  MUL_MAT_ID(type_a=UNKNOWN,type_b=F32,n_mats=4,n_used=2,b=0,m=512,n=1,k=256): not supported [CANN0] 
  MUL_MAT_ID(type_a=UNKNOWN,type_b=F32,n_mats=4,n_used=2,b=0,m=512,n=32,k=256): not supported [CANN0] 
  MUL_MAT_ID(type_a=UNKNOWN,type_b=F32,n_mats=4,n_used=2,b=0,m=512,n=1,k=256): not supported [CANN0] 
  MUL_MAT_ID(type_a=UNKNOWN,type_b=F32,n_mats=4,n_used=2,b=0,m=512,n=32,k=256): not supported [CANN0] 
  5473/5473 tests passed
  Backend CANN0: OK

Backend 2/2: CPU
  Skipping
2/2 backends passed
OK

TODO

The current matrix multiplication in MOE is quite slow. I’ll keep investing effort into this and look for a suitable aclnn acceleration operator as a replacement.

noemotiovon avatar Apr 21 '25 08:04 noemotiovon