Connection errors in production

[ianbbqzy]

What Happened?

Hi all, I'm occasionally running into several errors likely related to connection health in my production traffic. I can't reproduce locally even when I load test with >40 concurrent requests. I have observed these errors with openai, anthropic, and mistral. Curious if others have experienced the same or have any ideas how these could be reproduced/mitigated. Thanks in advance!

Failed to close the writer: anthropic TypeError [ERR_INVALID_STATE]: Invalid state: WritableStream is closed
    at writableStreamClose (node:internal/webstreams/writablestream:705:7)
    at writableStreamDefaultWriterClose (node:internal/webstreams/writablestream:1082:10)
    at WritableStreamDefaultWriter.close (node:internal/webstreams/writablestream:409:12)
    at file:///app/portkey-ai-gateway/build/start-server.js:2:359033
    at process.processTicksAndRejections (node:internal/process/task_queues:95:5)

What Should Have Happened?

Relevant Code Snippet

No response

Your Twitter/LinkedIn

No response