llm-inference topic
gpt4all
gpt4all: run open-source LLMs anywhere
ray-llm
RayLLM - LLMs on Ray
intel-extension-for-transformers
⚡ Build your chatbot within minutes on your favorite device; offer SOTA compression techniques for LLMs; run LLMs efficiently on Intel Platforms⚡
gateway
A Blazing Fast AI Gateway. Route to 100+ LLMs with 1 fast & friendly API.
PowerInfer
High-speed Large Language Model Serving on PCs with Consumer-grade GPUs
EcoAssistant
EcoAssistant: using LLM assistant more affordably and accurately
aibitat
Multi-Agent Conversation Framework in TypeScript
CompanionLLM
CompanionLLM - A framework to finetune LLMs to be your own sentient conversational companion
llmflows
LLMFlows - Simple, Explicit and Transparent LLM Apps
LLM-PowerHouse-A-Curated-Guide-for-Large-Language-Models-with-Custom-Training-and-Inferencing
LLM-PowerHouse: Unleash LLMs' potential through curated tutorials, best practices, and ready-to-use code for custom training and inferencing.