WhisperKit icon indicating copy to clipboard operation
WhisperKit copied to clipboard

Published 20 hours ago • verified publisher icon argmaxinc

Specialization takes a really long time

Open yych42 opened this issue 1 year ago • 1 comments

I'm trying the demo app on a MacBook Pro with Apple M1 Pro and 16 GB memory. The large-v3_turbo_1049MB model has been specializing for more than 30 minutes, but aned is still running and using a whole performance core. Have you guys tested the loading time on different devices?

yych42 avatar Feb 07 '24 12:02 yych42

Hi @yych42, It is a known issue with ANE compiler to take a very long time for turbo variants on A14 and M1 chips. We recently disabled turbo variants on devices with these chip generations but haven't updated the example app yet. In the interim, we recommend using regular large-v3.

atiorh avatar Feb 07 '24 16:02 atiorh

Following up here, there were a few updates in #20 that should help with this, but specializing is still a hard requirement from Apple that we don't have much control over via CoreML.

ZachNagengast avatar Feb 16 '24 21:02 ZachNagengast