WhisperKit
WhisperKit copied to clipboard
Implement test data-driven `unsupportedModelDeviceCombination` at init
After specifying a minimum OS version of macOS13 and iOS16, there is still a large matrix of possible model-device configurations for deployment:
Devices have varying capabilities across:
- Available RAM: Ranging from 4GB on iPhone 12 mini to 192GB on M2 Ultra Mac Studio
- Chip generation: A12 (iPhone XS) to A17 Pro (iPhone 15 Pro), M1-M3 for Mac and iPads etc. , S9 SiP for Apple Watch
- Apple Silicon vs legacy Intel Macs (still supported by macOS 14)
Model versions that have varying requirements:
- Original variants: tiny, base, small, large distil-large
- Argmax variants: compressed, turbo and compressed+turbo
Not all model versions will successfully run on all Apple devices. Some combinations may be enabled in the future by Argmax work. Some combinations may be disabled due to OS regressions. Hence, we would like to automatically test all combinations before each release using the testing infrastructure being laid out in #99 by @Abhinay1997 and throw unsupportedModelDeviceCombination
for combinations that fail to pass all the tests during WhisperKit
pipeline init time.
Taking it a step further, the WhisperKit
pipeline object should be able to take an optional "peakMemoryAllowed" argument at init time and compare that against the peak memory recorded during test on the particular model-device combination and throw unsupportedModelDeviceCombination
.
For the time being, we will disable all Intel Macs. Please reach out if you need Intel Macs support for your product.