cog-whisper icon indicating copy to clipboard operation
cog-whisper copied to clipboard

Support Large v2

Open anotherjesse opened this issue 2 years ago • 6 comments

https://github.com/openai/whisper/commit/4179ed2475cc84cba66868b516232ef1b74dacdf

OpenAI seems to have shipped a new version of the large model.

anotherjesse avatar Dec 07 '22 00:12 anotherjesse

  • The "large-v2" model is trained for more epochs with regularization and shows improved performance compared to the previous large.
  • It has the same architecture as the original large model.
  • When load_model("large") is called, the "large-v2" model will be loaded.
  • We will soon update the paper regarding this new model.

anotherjesse avatar Dec 07 '22 00:12 anotherjesse

just pushed a version supporting this :)

chenxwh avatar Dec 08 '22 22:12 chenxwh

Hi @chenxwh ,

Thanks for all your great work on this!

Just wanted to let you know that this new change is causing failures for models requesting the original "large" model that are set to use the latest model version.

Failing:

model = replicate.models.get("openai/whisper")
prediction = replicate.predictions.create(
        input={
                "audio": signed_audio_storage_link, 
                "model": "large", 
                "translate":False
            }, 
            version=model.versions.list()[0] # select latest model version
            )

Fix:

model = replicate.models.get("openai/whisper")
prediction = replicate.predictions.create(
        input={
                "audio": signed_audio_storage_link, 
                "model": "large-v2", 
                "translate":False
            }, 
            version=model.versions.list()[0] # select latest model version
            )

R-Gallagher avatar Dec 08 '22 22:12 R-Gallagher

yes now it is set to use large-v1 for the original and large-v2 for the new weights

chenxwh avatar Dec 08 '22 22:12 chenxwh

Ah, I see. I was using the above code in production and started getting errors, just left this here to help! I suppose I should pin the version number instead of always grabbing latest.

Cheers and thanks again!

R-Gallagher avatar Dec 08 '22 22:12 R-Gallagher

Will https://replicate.com/openai/whisper be updated with this?

ronyfadel avatar Dec 29 '22 13:12 ronyfadel