ml-cvnets
ml-cvnets copied to clipboard
apple
Run mobilevit-v2 failed with ANE(iphone 13 pro and ios15.5)
Hi, with iphone13pro and ios15.5，i got the error when running mobilevit-v2. Do you have any experience to deal with it?
2022-06-27 15:12:57.196279+0800 CoreMLPerformance[10320:18583922] [espresso] [Espresso::handle_ex_plan] exception=ANECF error: ANECCompile(/var/mobile/Library/Caches/com.apple.aned/tmp/com.example.CoreMLPerformance/824F1B79C0CBD30CBE0EFD39FDD54AB15B3E93F859264B1FE69FDB3F02C8D241/C5992495EF614BA515A970B3EBE95F193A7C62E4F7478D853BED6F250D40E299/) FAILED: err=( CompilationFailure ) 2022-06-27 15:12:57.196809+0800 CoreMLPerformance[10320:18583922] [coreml] Error plan build: -1.
The mobilevit-v2 is produced by following example command. export CONFIG_FILE="https://docs-assets.developer.apple.com/ml-research/models/cvnets-v2/classification/mobilevitv2/imagenet1k/256x256/mobilevitv2-1.0.yaml" export MODEL_WEIGHTS="https://docs-assets.developer.apple.com/ml-research/models/cvnets-v2/classification/mobilevitv2/imagenet1k/256x256/mobilevitv2-1.0.pt" cvnets-convert --common.config-file $CONFIG_FILE --common.results-loc coreml_models_cls --model.classification.pretrained $MODEL_WEIGHTS --conversion.coreml-extn mlmodel
Hi @zshanwei ,
It is hard to say what went wrong with this limited information, but I think, one potential idea to start debugging is using the network graph visualization. Could you please visualize the converted CoreML model with Netron and share the output graph?
Also, what version of Coremltools and MacBook that you are using for conversion?
Hi @sacmehta,
I exported it with coremltools 5.2.0 and successfully tested it with the new Xcode beta Core ML performance monitor but it looks like only 81 ops are running on the ANE and 348 on the CPU. I'm also having a min latency of 26.93ms on an iPhone11 running iOS 16 beta 2.
Is this the expected behavior ?
hi @sacmehta
Coremltools version==5.2.0.
MacBook system version==12.4
the mobilevit-v2 model produced by example command is as follow.
The converted graph looks good to me. I am not sure about the error that you are getting and I think it would be better if you post this error on CoreML page. They would be able to assist you better with this error.
Also, did you try after updating the coremltools package?
@JacopoMangiavacchi I did not test on iPhone 11. It is possible that iPhone 11 does not support some operations and are mapped to CPU. It would be good to check it with CoreMLtools team.
The converted graph looks good to me. I am not sure about the error that you are getting and I think it would be better if you post this error on CoreML page. They would be able to assist you better with this error.
Also, did you try after updating the coremltools package?
Hi @sacmehta , i have tried with 5.2.0 and 6.0b1. The converted MobileVit-V2 mlmodel stiil runs failed with ANECF exception error.
Hi @sacmehta By analyzing the model, i have found that the problem is caused by dim value in torch.sum at https://github.com/apple/ml-cvnets/blob/main/cvnets/layers/linear_attention.py#L157 If dim is -1, the converted mlmodel will fail with ANECF exception. If dim is set to 2, it will work normally. I think the reason is coreml model IR's compatibility for axes attribute in reduceSum op when executed by ANE.
Besides, when I tested the right model with iphone 13 pro and ios15.5, the latency on ane is 5.2ms. I follow the method by https://github.com/vladimir-chernykh/coreml-performance. However, the latency in paper is 3.4ms for MobileViTv2-1.0 on iphone12. So could you please provide your performance testing tool or method to align with your result？