gateway
gateway copied to clipboard
Fix/unhandled exception in groq streaming requests
Author Description
This PR fixes the #1072 and enable the gateway to process groq completion requests when stream_options is used.
Summary By MatterAI
๐ What Changed
This PR fixes an unhandled exception in Groq streaming requests by adding proper null checks and handling undefined values in the stream response. The changes focus on the GroqChatCompleteStreamChunkTransform function to properly handle edge cases when processing stream chunks from Groq API.
๐ Impact of the Change
The fix enables the gateway to properly process Groq completion requests when stream_options is used, resolving issue #1072. This prevents crashes and improves reliability when streaming responses from Groq.
๐ Total Files Changed
1 file modified: src/providers/groq/chatComplete.ts with 34 additions and 21 deletions.
๐งช Test Added
N/A - No tests were added in this PR.
๐ Security Vulnerabilities
N/A - No security vulnerabilities were addressed in this PR.
Type of Change
- [x] Bug fix (non-breaking change which fixes an issue)
- [ ] New feature (non-breaking change which adds functionality)
- [ ] Breaking change (fix or feature that would cause existing functionality to not work as expected)
- [ ] Documentation update
- [ ] Refactoring (no functional changes)
How Has This Been Tested?
- [ ] Unit Tests
- [ ] Integration Tests
- [x] Manual Testing
Screenshots (if applicable)
N/A
Checklist
- [x] My code follows the style guidelines of this project
- [x] I have performed a self-review of my own code
- [ ] I have commented my code, particularly in hard-to-understand areas
- [ ] I have made corresponding changes to the documentation
- [x] My changes generate no new warnings
- [ ] I have added tests that prove my fix is effective or that my feature works
- [ ] New and existing unit tests pass locally with my changes
Related Issues
Fixes #1072
Quality Recommendations
-
Add unit tests to verify the fix for streaming requests
-
Consider adding more comprehensive error handling for other potential edge cases in the stream response
Sequence Diagram
sequenceDiagram
participant Client
participant Gateway
participant GroqAPI
Client->>Gateway: Send streaming chat completion request
Gateway->>GroqAPI: Forward request with stream_options
GroqAPI-->>Gateway: Return stream chunks
loop For each stream chunk
Gateway->>Gateway: GroqChatCompleteStreamChunkTransform()
Note over Gateway: Handle potential undefined values
Note over Gateway: Add proper null checks for choices
Note over Gateway: Map usage data correctly
end
Gateway-->>Client: Return transformed stream chunks
LGTM! ๐๐๐๐