gateway icon indicating copy to clipboard operation
gateway copied to clipboard

Published 20 hours ago • verified publisher icon Portkey-AI

How to integrate locally deployed models, such as the Qwen model deployed with vLLM, with an AI gateway?

Open wgimperial opened this issue 1 year ago • 8 comments

What Happened?

I would like to use AI Gateway to manage a series of locally deployed models, but I'm not sure how to proceed.

What Should Have Happened?

No response

Relevant Code Snippet

No response

Your Twitter/LinkedIn

No response

wgimperial avatar Sep 20 '24 08:09 wgimperial