gateway
gateway copied to clipboard
Support Vertex Llama Hosted API Enpoint
Title:
- Support Llama hosted API Model on Vertex https://console.cloud.google.com/vertex-ai/publishers/meta/model-garden/llama3-405b-instruct-maas
Description: (optional)
- Created transform and config for llama inside vertex
- Handled JSON streaming response
Related Issues: (optional)
- #485
Tests Done:
- Tested streaming for different providers on vertex
- Tested streaming for google and anthropic and vertex google and vertex llama