whisper-jax icon indicating copy to clipboard operation
whisper-jax copied to clipboard

Published 20 hours ago • verified publisher icon sanchit-gandhi

Can this be deployed on replicate.com?

Open troublesprouter opened this issue 1 year ago • 6 comments

Has anyone deployed it on replicate.com?

I want to use the model via an API but only paying per second of use. Not pay to keep a server running at all times.

Any ideas on how to achieve that on replicate.com or elsewhere, please let me know.

troublesprouter avatar Jun 14 '23 14:06 troublesprouter

You can use the Hugging Face one for free as far as I'm aware, the queue is usually always 0-2.

CoocooFroggy avatar Jun 14 '23 16:06 CoocooFroggy

I need to deploy a faster version of my own for use in production via Hugging Face, but when I try to duplicate the space to change the hardware, I get this API_KEY setting that I don't know what it is:

image

And if you leave it blank it just fails to build.

Any idea on how to set it or otherwise on how to deploy whisper-jax in a high performance environemnt that only charges for use time?

troublesprouter avatar Jun 14 '23 17:06 troublesprouter

Hey! Have you implement any of these so far? Any updates?

aqasemi avatar Jul 28 '23 18:07 aqasemi

Perhaps the Hugging Face space is just a proxy to the real back end, hence the API_URL secret? Just a guess

CoocooFroggy avatar Jul 28 '23 21:07 CoocooFroggy

@troublesprouter I've managed to deploy it on replicate. But it wasn't as fast as expected because replicate doesn't support TPUs. However, I tried to deploy faster-whisper and it was blazingly fast.

https://replicate.com/alqasemy2020/whisper-jax ignore its name (it's not whisper-jax) ( :

aqasemi avatar Jul 29 '23 00:07 aqasemi

If you want to duplicate the hugging face version have a look at this https://huggingface.co/spaces/sanchit-gandhi/whisper-jax/discussions/38#648b03a7084699ca1535eddc

aqasemi avatar Jul 29 '23 00:07 aqasemi