spring-ai
spring-ai copied to clipboard
fix: Add instructions support for OpenAI TTS models
trafficstars
Description
Add optional instructions parameter to OpenAI TTS models to support style/tone guidance for speech synthesis. This addresses the need to control prosody, emotion, and delivery style for models that support it (currently gpt-4o-mini-tts).
Fixes #4388
Changes
Core Implementation
- Add
instructionsfield toOpenAiAudioSpeechOptionswith getter/setter and builder support - Add
instructionsparameter toOpenAiAudioApi.SpeechRequestrecord - Implement conditional passing of instructions based on model whitelist
- Add
INSTRUCTIONS_SUPPORTED_MODELSstatic set (currently containsgpt-4o-mini-tts) - Log warning for unsupported models instead of throwing exception
- Update
equals(),hashCode(),toString(), andcopy()methods to include instructions
Testing
- Add
OpenAiAudioSpeechModelInstructionsTeststo verify conditional instruction passing - Add
OpenAiSpeechRequestInstructionsSerializationTeststo verify JSON serialization
Documentation
- Update
openai-speech.adocwith:- Configuration property for
spring.ai.openai.audio.speech.options.instructions - Runtime options example showing instructions usage
- New section “Using Instructions for Style Control” with implementation details
- Important note about model compatibility
- Configuration property for
Backward Compatibility
Fully backward compatible:
instructionsparameter is optional (nullable)- All existing constructors and methods remain unchanged
- Unsupported models gracefully ignore instructions with a warning log
- No breaking changes to API or configuration
Usage Example
OpenAiAudioSpeechOptions options = OpenAiAudioSpeechOptions.builder()
.model("gpt-4o-mini-tts")
.voice(OpenAiAudioApi.SpeechRequest.Voice.VERSE)
.instructions("Friendly; warm tone; natural pauses")
.build();
TextToSpeechPrompt prompt = new TextToSpeechPrompt("Welcome!", options);
TextToSpeechResponse response = model.call(prompt);
Checklist
- [x] Add a Signed-off-by line to each commit (
git commit -s) - [x] Rebase changes on the latest
mainbranch and squash commits - [x] Add/Update unit tests
- [x] Run the build and ensure all tests pass (
mvn test) - [x] Update documentation if necessary
- [x] Ensure backward compatibility
- [x] Follow existing code style and conventions