LibreChat icon indicating copy to clipboard operation
LibreChat copied to clipboard

🎤 feat: add custom speech config, browser TTS/STT features, and dynamic speech tab settings

Open berry-13 opened this issue 8 months ago • 5 comments

Summary

This PR introduces several key features and improvements related to the speech functionality in LibreChat:

  • Custom Speech Configuration: Added a custom speech configuration option in librechat.yaml, allowing the ADMIN to set pre-configured "speech tab" settings
  • Browser TTS Language Selection: Enabled language selection for text-to-speech (TTS) directly within the browser
  • Browser STT Streaming: Implemented streaming for speech-to-text (STT) in the browser
  • Dynamic Speech Tab Settings: The speech tab settings now dynamically appear and disappear based on user settings (e.g., browser/external engine dropdown)
  • Refactoring:
    • Renamed endpointSTT and endpointTTS to engineSTT and engineTTS respectively
    • Moved the speech API to a subpath /api/files/speech

Breaking Changes

  • The variables SpeechToText and TextToSpeech in the store have been renamed to speechToText and textToSpeech. If you encounter any issues, please delete LibreChat's cache
  • The speech section has been moved under the speech:

Change Type

  • [x] New feature (non-breaking change which adds functionality)
  • [x] This change requires a documentation update

Testing

External STT/TTS:

  • ElevenLabs
  • OpenAI

Local STT/TTS:

  • Chrome

Test Configuration:

To reproduce the test process, follow these steps:

  1. Configure the librechat.yaml file with appropriate speech settings
  2. Select a TTS language in the browser and verify the speech output
  3. Test STT streaming functionality in the browser using various input sources
  4. Verify that the speech tab settings appear or disappear based on the selected user settings
  5. Ensure all refactored endpoints (engineSTT and engineTTS) function correctly
  6. Confirm the speech API is accessible via the new subpath /api/files/speech

Checklist

  • [x] My code adheres to this project's style guidelines.
  • [x] I have performed a self-review of my own code.
  • [x] I have commented in any complex areas of my code.
  • [x] I have made pertinent documentation changes.
  • [x] My changes do not introduce new warnings.
  • [x] I have written tests demonstrating that my changes are effective or that my feature works.
  • [x] Local unit tests pass with my changes.
  • [x] Any changes dependent on mine have been merged and published in downstream modules.
  • [x] A pull request for updating the documentation has been submitted.

berry-13 avatar May 30 '24 15:05 berry-13