llm-inference topic
gpt4all
GPT4All: Run Local LLMs on Any Device. Open-source and available for commercial use.
ray-llm
RayLLM - LLMs on Ray
intel-extension-for-transformers
⚡ Build your chatbot within minutes on your favorite device; offer SOTA compression techniques for LLMs; run LLMs efficiently on Intel Platforms⚡
gateway
A blazing fast AI Gateway with integrated guardrails. Route to 200+ LLMs, 50+ AI Guardrails with 1 fast & friendly API.
PowerInfer
High-speed Large Language Model Serving on PCs with Consumer-grade GPUs
EcoAssistant
EcoAssistant: using LLM assistant more affordably and accurately
aibitat
Multi-Agent Conversation Framework in TypeScript
CompanionLLM
CompanionLLM - A framework to finetune LLMs to be your own sentient conversational companion
llmflows
LLMFlows - Simple, Explicit and Transparent LLM Apps
LLM-PowerHouse-A-Curated-Guide-for-Large-Language-Models-with-Custom-Training-and-Inferencing
LLM-PowerHouse: Unleash LLMs' potential through curated tutorials, best practices, and ready-to-use code for custom training and inferencing.