gateway icon indicating copy to clipboard operation
gateway copied to clipboard

Fix/unhandled exception in groq streaming requests

Open metinogurlu opened this issue 7 months ago โ€ข 1 comments

Code Quality bug fix

Author Description

This PR fixes the #1072 and enable the gateway to process groq completion requests when stream_options is used.

Summary By MatterAI

๐Ÿ”„ What Changed

This PR fixes an unhandled exception in Groq streaming requests by adding proper null checks and handling undefined values in the stream response. The changes focus on the GroqChatCompleteStreamChunkTransform function to properly handle edge cases when processing stream chunks from Groq API.

๐Ÿ” Impact of the Change

The fix enables the gateway to properly process Groq completion requests when stream_options is used, resolving issue #1072. This prevents crashes and improves reliability when streaming responses from Groq.

๐Ÿ“ Total Files Changed

1 file modified: src/providers/groq/chatComplete.ts with 34 additions and 21 deletions.

๐Ÿงช Test Added

N/A - No tests were added in this PR.

๐Ÿ”’ Security Vulnerabilities

N/A - No security vulnerabilities were addressed in this PR.

Type of Change

  • [x] Bug fix (non-breaking change which fixes an issue)
  • [ ] New feature (non-breaking change which adds functionality)
  • [ ] Breaking change (fix or feature that would cause existing functionality to not work as expected)
  • [ ] Documentation update
  • [ ] Refactoring (no functional changes)

How Has This Been Tested?

  • [ ] Unit Tests
  • [ ] Integration Tests
  • [x] Manual Testing

Screenshots (if applicable)

N/A

Checklist

  • [x] My code follows the style guidelines of this project
  • [x] I have performed a self-review of my own code
  • [ ] I have commented my code, particularly in hard-to-understand areas
  • [ ] I have made corresponding changes to the documentation
  • [x] My changes generate no new warnings
  • [ ] I have added tests that prove my fix is effective or that my feature works
  • [ ] New and existing unit tests pass locally with my changes

Related Issues

Fixes #1072

Quality Recommendations

  1. Add unit tests to verify the fix for streaming requests

  2. Consider adding more comprehensive error handling for other potential edge cases in the stream response

Sequence Diagram

sequenceDiagram
    participant Client
    participant Gateway
    participant GroqAPI
    
    Client->>Gateway: Send streaming chat completion request
    Gateway->>GroqAPI: Forward request with stream_options
    GroqAPI-->>Gateway: Return stream chunks
    
    loop For each stream chunk
        Gateway->>Gateway: GroqChatCompleteStreamChunkTransform()
        Note over Gateway: Handle potential undefined values
        Note over Gateway: Add proper null checks for choices
        Note over Gateway: Map usage data correctly
    end
    
    Gateway-->>Client: Return transformed stream chunks

metinogurlu avatar Apr 30 '25 11:04 metinogurlu

LGTM! ๐Ÿš€๐Ÿš€๐Ÿš€๐Ÿš€

narengogi avatar Apr 30 '25 11:04 narengogi