Hello sanchit-gandhi, and first of all, thank you for the tremendous work on the Whisper JAX model - its speed is HUGE, and it is really efficient with the architecture you put in place.
I've been building using the endpoint you provide, but today, it seems I cannot access it anymore.
Full log of the error I receive :
2023-09-10T10:04:43.059627+00:00 app[worker.1]: Traceback (most recent call last):
2023-09-10T10:04:43.059627+00:00 app[worker.1]: File "/app/src/whisper_jax_endpoint.py", line 25, in
2023-09-10T10:04:43.059627+00:00 app[worker.1]: output = transcribe_audio(audio_path, return_timestamps=True)
2023-09-10T10:04:43.059627+00:00 app[worker.1]: ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2023-09-10T10:04:43.059628+00:00 app[worker.1]: File "/app/src/whisper_jax_endpoint.py", line 10, in transcribe_audio
2023-09-10T10:04:43.059628+00:00 app[worker.1]: text, runtime = client.predict(
2023-09-10T10:04:43.059629+00:00 app[worker.1]: ^^^^^^^^^^^^^^^
2023-09-10T10:04:43.059629+00:00 app[worker.1]: File "/app/.heroku/python/lib/python3.11/site-packages/gradio_client/client.py", line 291, in predict
2023-09-10T10:04:43.059629+00:00 app[worker.1]: return self.submit(*args, api_name=api_name, fn_index=fn_index).result()
2023-09-10T10:04:43.059635+00:00 app[worker.1]: ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2023-09-10T10:04:43.059636+00:00 app[worker.1]: File "/app/.heroku/python/lib/python3.11/site-packages/gradio_client/client.py", line 1120, in result
2023-09-10T10:04:43.059636+00:00 app[worker.1]: return super().result(timeout=timeout)
2023-09-10T10:04:43.059636+00:00 app[worker.1]: ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2023-09-10T10:04:43.059637+00:00 app[worker.1]: File "/app/.heroku/python/lib/python3.11/concurrent/futures/_base.py", line 456, in result
2023-09-10T10:04:43.059638+00:00 app[worker.1]: return self.__get_result()
2023-09-10T10:04:43.059638+00:00 app[worker.1]: ^^^^^^^^^^^^^^^^^^^
2023-09-10T10:04:43.059638+00:00 app[worker.1]: File "/app/.heroku/python/lib/python3.11/concurrent/futures/_base.py", line 401, in __get_result
2023-09-10T10:04:43.059638+00:00 app[worker.1]: raise self._exception
2023-09-10T10:04:43.059639+00:00 app[worker.1]: File "/app/.heroku/python/lib/python3.11/concurrent/futures/thread.py", line 58, in run
2023-09-10T10:04:43.059639+00:00 app[worker.1]: result = self.fn(*self.args, **self.kwargs)
2023-09-10T10:04:43.059639+00:00 app[worker.1]: ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2023-09-10T10:04:43.059639+00:00 app[worker.1]: File "/app/.heroku/python/lib/python3.11/site-packages/gradio_client/client.py", line 793, in _inner
2023-09-10T10:04:43.059639+00:00 app[worker.1]: predictions = _predict(*data)
2023-09-10T10:04:43.059640+00:00 app[worker.1]: ^^^^^^^^^^^^^^^
2023-09-10T10:04:43.059640+00:00 app[worker.1]: File "/app/.heroku/python/lib/python3.11/site-packages/gradio_client/client.py", line 822, in _predict
2023-09-10T10:04:43.059640+00:00 app[worker.1]: result = utils.synchronize_async(self._ws_fn, data, hash_data, helper)
2023-09-10T10:04:43.059640+00:00 app[worker.1]: ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2023-09-10T10:04:43.059640+00:00 app[worker.1]: File "/app/.heroku/python/lib/python3.11/site-packages/gradio_client/utils.py", line 527, in synchronize_async
2023-09-10T10:04:43.059641+00:00 app[worker.1]: return fsspec.asyn.sync(fsspec.asyn.get_loop(), func, *args, **kwargs) # type: ignore
2023-09-10T10:04:43.059641+00:00 app[worker.1]: ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2023-09-10T10:04:43.059641+00:00 app[worker.1]: File "/app/.heroku/python/lib/python3.11/site-packages/fsspec/asyn.py", line 106, in sync
2023-09-10T10:04:43.059641+00:00 app[worker.1]: raise return_result
2023-09-10T10:04:43.059642+00:00 app[worker.1]: File "/app/.heroku/python/lib/python3.11/site-packages/fsspec/asyn.py", line 61, in _runner
2023-09-10T10:04:43.059642+00:00 app[worker.1]: result[0] = await coro
2023-09-10T10:04:43.059642+00:00 app[worker.1]: ^^^^^^^^^^
2023-09-10T10:04:43.059642+00:00 app[worker.1]: File "/app/.heroku/python/lib/python3.11/site-packages/gradio_client/client.py", line 1050, in _ws_fn
2023-09-10T10:04:43.059642+00:00 app[worker.1]: return await utils.get_pred_from_ws(websocket, data, hash_data, helper)
2023-09-10T10:04:43.059642+00:00 app[worker.1]: ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2023-09-10T10:04:43.059643+00:00 app[worker.1]: File "/app/.heroku/python/lib/python3.11/site-packages/gradio_client/utils.py", line 273, in get_pred_from_ws
2023-09-10T10:04:43.059643+00:00 app[worker.1]: msg = task.result()
2023-09-10T10:04:43.059643+00:00 app[worker.1]: ^^^^^^^^^^^^^
2023-09-10T10:04:43.059643+00:00 app[worker.1]: File "/app/.heroku/python/lib/python3.11/site-packages/websockets/legacy/protocol.py", line 568, in recv
2023-09-10T10:04:43.059644+00:00 app[worker.1]: await self.ensure_open()
2023-09-10T10:04:43.059644+00:00 app[worker.1]: File "/app/.heroku/python/lib/python3.11/site-packages/websockets/legacy/protocol.py", line 935, in ensure_open
2023-09-10T10:04:43.059644+00:00 app[worker.1]: raise self.connection_closed_exc()
2023-09-10T10:04:43.059644+00:00 app[worker.1]: websockets.exceptions.ConnectionClosedError: received 1011 (unexpected error) keepalive ping timeout; then sent 1011 (unexpected error) keepalive ping timeout
Any idea where that may come from ? I also cannot use the website version of the endpoint...
Last, since I plan on working on a professional tool based on your model, could we exchange on providing a more private / focused endpoint I could access with my tool to make sure it is optimized ?
Hope to exchange with you soon !
I don't know if you are still waiting for a response from the owner of the repository, but if you are looking for a new solution or something to replace it in the meantime then I have a recommendation.
https://github.com/djmango/obsidian-transcription
It has a server side set up for a more professional model like you were looking for, or you can just use the website linked to the github. I haven't tested this Whisper JAX model before, but Obsidian Transcription is also fast and accurate. I would give it a try if you aren't able to find a solution.
I may indeed... Is there a diarization to the model ? I didn't see it yet in the Readme. Thanks for the recommandation anyways !
Not yet, however diarization seems to be planned soon. I'll link the API documents below so you can see if anything catches your eye.
swiftink.io/docs
No problem!
whisper-jax copied to clipboard
sanchit-gandhi
