omi icon indicating copy to clipboard operation
omi copied to clipboard

add elevenlabs scribe

Open neooriginal opened this issue 2 months ago • 5 comments

I did also implement backend/scripts/stt/k_compare_transcripts_performance.py but i could not test it yet.

This pull request introduces ElevenLabs Scribe as a new speech-to-text (STT) integration, making it available as a selectable transcription service alongside existing providers. The changes span environment configuration, requirements, core streaming logic, database storage, performance testing, and include a new integration test script. The main updates focus on supporting ElevenLabs Scribe throughout the backend pipeline, enabling its use for real-time and batch transcription.

ElevenLabs Scribe Integration

  • Added the elevenlabs Python package to requirements.txt and the ELEVENLABS_API_KEY environment variable to .env.template for authentication and configuration. [1] [2]
  • Implemented ElevenLabs Scribe support in the streaming pipeline (process_audio_elevenlabs), updated service selection logic to prioritize ElevenLabs when configured, and mapped supported languages. [1] [2] [3]
  • Integrated ElevenLabs Scribe into the main transcription router, including socket management, audio streaming, and resource cleanup. [1] [2] [3] [4] [5] [6] [7] [8]

Database and Performance Testing

  • Updated the database logic to store and retrieve ElevenLabs Scribe transcripts, ensuring results are available alongside other providers.
  • Enhanced the transcript performance comparison script to include ElevenLabs, allowing for benchmarking and validation against other STT services. [1] [2] [3] [4] [5]

Testing and Validation

  • Added a standalone integration test script (test_elevenlabs_stt.py) to verify ElevenLabs Scribe connectivity, service selection, and basic audio streaming functionality.

neooriginal avatar Oct 23 '25 20:10 neooriginal

I cannot merge if the pr was not tested. pls get back to me with the test results.

this pr is clean. nice job. thanks man.

beastoin avatar Nov 05 '25 02:11 beastoin

I cannot merge if the pr was not tested. pls get back to me with the test results.

this pr is clean. nice job. thanks man.

It was fully tested, don't worry. I've been running it on my custom backend for a few weeks now.

neooriginal avatar Nov 05 '25 05:11 neooriginal

I can't sadly test backend/scripts/stt/k_compare_transcripts_perfor mance.py as I don't have the funds to test all of then. If you want me to test it, please get back to me on discord with the elevenlabs API key.

neooriginal avatar Nov 05 '25 07:11 neooriginal

Thank you all on your support

Benson665 avatar Nov 05 '25 13:11 Benson665

@beastoin

neooriginal avatar Nov 07 '25 11:11 neooriginal

Scribe v2 is real-time but does not support diarization, scribe v1 supports diarization but is not real-time

mdmohsin7 avatar Dec 01 '25 17:12 mdmohsin7

Scribe v2 is real-time but does not support diarization, scribe v1 supports diarization but is not real-time

Thats a good point, i did not check before migrating to v2. Sorry about that. Want me to revert back to v1? Deepgram is terrible

neooriginal avatar Dec 01 '25 17:12 neooriginal

Also there are a few issues in code, elevenlabs_socket, elevenlabs_socket1, elevenlabs_socket2, you'd probably want to replace elevenlabs_socket and its instances with elevenlabs_socket1

mdmohsin7 avatar Dec 01 '25 18:12 mdmohsin7

Also there are a few issues in code, elevenlabs_socket, elevenlabs_socket1, elevenlabs_socket2, you'd probably want to replace elevenlabs_socket and its instances with elevenlabs_socket1

Got it, will it be merged if fixed?

neooriginal avatar Dec 01 '25 18:12 neooriginal