✨ instinct.cpp

instinct.cpp is a toolkit for developing LLM-powered applications.

🚨 This project is under active development and has not reached to GA stage of first major release. See more at Roadmap section.

Features

Components of instinct.cpp

What instinct.cpp offer:

Applications that are working out-of-box.
- Assistant API server: Agent service that is fully compatible with OpenAI's Assistant API.
  - mini-assistant-api: Single binary for single node deployment with vector database and other dependencies bundled.
  - mighty-assistant-api: (WIP) A cloud native implementation that is highly scalable with distributed components and multi-tenant support.
- chat-agent: A CLI application that create knowledge index with your docs (PDF,TXT,MD,...) and launch an HTTP server that is fully compatible with OpenAI ChatCompletion.
Frameworks to build LLM-based applications. Say it langchain.cpp.
- Integration for privacy-first LLM providers: Built-in support for Ollama and other OpenAI compatible API services like vllm, llama.cpp server, nitro and more.
- Building blocks for common application patterns like Chatbot, RAG, LLM Agent.
- Functional chaining components for composable LLM pipelines.
- Agent patterns: ReACT, OpenAI-based tool agent, LLMCompiler, ...

For built-in applications:

For library itself:

Complete project plan is tracked at Project kanban.

Milestone	Features	DDL
v0.1.0	Long-short memory, PDF/TXT/DOCX ingestor, `Chain` programing paradigm, RAG reference app `doc-agent`	3.29
v0.1.1	Performance tuning, RAG evaluation, Function calling agent	4.16
v0.1.2	OpenAI Assistant API initial implementation, single-binary reference app `mini-assistant`	4.30
v0.1.3	* `mini-assistant`: tool calls with opensourced LLMs	5.17
v0.1.4	* `doc-agent` : rerank model * `mini-assistant`: `file-search` tool support.	~~6.18~~ 6.14
v0.1.5	Overall optimization	6.30
v0.1.6	`code-interpreter` in `mini-assistant`	7.15

Contributions are welcomed! You can join discord server, or contact me via email.