Core ML perf regression observed on non-genai models

[guangy10]

🐛 Describe the bug

We are running exercise on the newly launched benchmarking infra with the in-tree enabled models under examples/models. You can follow the instruction to extract and inspect the benchmark_results.json. (Connecting the results to a dashboard is still WIP)

The preliminary data points collected from the benchmarking infra can be found here: https://github.com/pytorch/executorch/tree/main/extension/benchmark#preliminary-benchmark-results. It shows some previously enabled models, e.g. MobileNetV2/3, InceptionV3/4, are running expected slower with Core ML delegates.

The models are exported using the coreml export scripts (code pointer), and the app that loads and runs the exported model is located under extension/apple/benchmark.

Versions

From latest executorch main