gateway
gateway copied to clipboard
A blazing fast AI Gateway with integrated guardrails. Route to 200+ LLMs, 50+ AI Guardrails with 1 fast & friendly API.
 Snyk has created this PR to upgrade next from 14.2.14 to 14.2.29. :information_source: Keep your dependencies up-to-date. This makes it easier to fix existing vulnerabilities and to more quickly...
 Snyk has created this PR to upgrade ai from 3.4.9 to 3.4.33. :information_source: Keep your dependencies up-to-date. This makes it easier to fix existing vulnerabilities and to more quickly...
 Snyk has created this PR to upgrade tailwind-merge from 2.5.3 to 2.6.0. :information_source: Keep your dependencies up-to-date. This makes it easier to fix existing vulnerabilities and to more quickly...
 Snyk has created this PR to upgrade lucide-react from 0.366.0 to 0.513.0. :information_source: Keep your dependencies up-to-date. This makes it easier to fix existing vulnerabilities and to more quickly...
 Snyk has created this PR to upgrade zod from 3.23.8 to 3.25.51. :information_source: Keep your dependencies up-to-date. This makes it easier to fix existing vulnerabilities and to more quickly...
### What Happened? When using Bedrock with `cache_control: "ephemeral"` on tool use content blocks, the returned Bedrock data has no `cache_read_input_tokens` or `cache_creation_input_tokens`. It seems to only affect tool use...
## Description ## Motivation ## Type of Change - [ ] Bug fix (non-breaking change which fixes an issue) - [ ] New feature (non-breaking change which adds functionality) -...
### What Would You Like to See with the Gateway? Environment Details: Portkey Version: portkeyai/gateway:1.10.0 (public Docker image) Deployment: Self-hosted in a Kubernetes environment. Goal: To configure a virtual key...
Environment Details: Portkey Version: portkeyai/gateway:1.10.0 (public Docker image) Deployment: Self-hosted in a Kubernetes environment. Backend Service: NVIDIA Triton Inference Server Models: BAAI/bge-large-en-v1.5 (Embedding) BAAI/bge-reranker-large (Reranker) Goal: To use the self-hosted...
## Description This PR improves token usage tracking in Google Vertex AI streaming responses by storing prompt token information in the stream state and including token usage metrics in response...