JetStream icon indicating copy to clipboard operation
JetStream copied to clipboard

Published 20 hours ago • verified publisher icon google
→

Metadata

JetStream is a throughput and memory optimized engine for LLM inference on XLA devices, starting with TPUs (and GPUs in future -- PRs welcome).

Results 21 JetStream issues
Sort by recently updated
recently updated
newest added

Support completions API

Supporting openai-api-compatible endpoints like the `/v1/chat/completions` and `/v1/completions` APIs would have the following benefits: * Allow Jetstream to be used as a drop-in replacement for the vLLM server * Make...

nstogner avatar nstogner

← Metadata

196
Stars
26
Forks
Watchers

Owner

verified publisher icon google

Metadata

JetStream is a throughput and memory optimized engine for LLM inference on XLA devices, starting with TPUs (and GPUs in future -- PRs welcome).

Back