WhisperKit icon indicating copy to clipboard operation
WhisperKit copied to clipboard

`detectLanguage` isn't working

Open stasbel opened this issue 8 months ago • 5 comments

version: 0.7.2 snippet to reproduce:

import Foundation
import WhisperKit

Task {
  guard let desktopURL = FileManager.default.urls(
    for: .desktopDirectory,
    in: .userDomainMask
  ).first else { return }
  print("desktopURL: \(desktopURL)")
  
  let model = try await WhisperKit(model: "base")
  try await model.loadModels(prewarmMode: true)
  
  let fileURL = desktopURL.appendingPathComponent("0.wav")
  print("absoluteString: \(fileURL.path)")
  let (lang, langProbs) = try await model.detectLanguage(audioPath: fileURL.path)
  print("lang: \(lang), langProbs: \(langProbs)")
}

file: 0.wav.zip

tiny version gives:

lang: en, langProbs: ["en": -0.35910118]"

which is not all languages and has negative prob

base version gives:

[WhisperKit] Detected language nocaptions is not supported, defaulting to en
lang: en, langProbs: [:]

which is just plain error somewhere inside whisperkit

stasbel avatar Jun 17 '24 15:06 stasbel