t41372
[IDEA] 有关于插件系统的问题。Questions about the plugin system.
此前我在 QQ 群和 Zulip 社区中曾分享过下一代架构也就是OLV2的初步设计图。考虑到当前社区对可扩展功能日益增长的需求，同时对比 SuperAgentParty、吟美、小智等新兴数字人平台，我们的架构在灵活性与可控性上具备独特优势，因此我想借此机会重新提出这一构想，并邀请大家共同探讨。
附上此前设计的架构图供大家参考。整体设计思路是：为追求功能实现的简洁与优雅，建议将插件系统仅用于“可选拓展模块”，而核心/基础功能仍以直接代码更新（硬更新）为主。这样既能保证系统稳定性，又能支持灵活扩展。 我做了一个插件系统的设图。
我们可考虑引入类似“前端触发后端更新”的机制（类似 OTA 升级），或构建一个“插件市场”，让终端用户按需选择并安装自己偏好的应用模块 —— 这样既降低使用门槛，又提升个性化体验。
接下来需要重点讨论的是：哪些功能适合插件化？哪些应保留在核心层？欢迎大家提出意见，一起完善这个架构方向。
I previously shared a preliminary architecture diagram for our next-generation system in both the QQ group and Zulip community. Given the growing community demand for extensible features — and considering how our design holds distinct advantages in flexibility and control compared to emerging digital human platforms like SuperAgentParty, Yinmei, and XiaoZhi — I’d like to revisit this concept and invite everyone to join the discussion.
Attached is the earlier version of the architecture diagram for your reference. The core design philosophy is this: to maintain elegance and simplicity, we recommend that the plugin system be used only for optional, extendable modules. Core/base functionalities should still be updated via direct code changes (“hard updates”) to ensure system stability while enabling flexible expansion.
We could explore mechanisms such as “frontend-triggered backend updates” (similar to OTA-style upgrades), or even build a “plugin marketplace” where end users can freely browse, select, and install their preferred application modules — lowering adoption barriers while enhancing personalization.
The key question we now need to discuss together is: Which features should be plugin-ized, and which should remain part of the core system? Your insights and suggestions are warmly welcome as we refine this architectural direction.
Would it be possible to have an English version of the architecture diagram? I don't know how to read Mandarin 😢
@theneur0mancer the mind map sofware cant store data and the mindmap had been missing. here is some of explan i use AI to show to you:
Open-LLM-VTuber: Functions to Be Added
-
Plugin & Extension System (Green Branch) Plugin Hot Loading Plugin Store Custom Plugin Development Framework Plugin Permission Management Plugin Configuration Interface Difference between Plugins and MCP: Plugins can be more seamlessly integrated into OLV's logic. Example Plugins Social Platform Integration Features QQ Robot Integration WeChat Robot Integration Discord/Telegram Integration Online "Rage Mode": Allows OLV character to engage in online trolling Multimedia Control Features Music Playback Control Playlist Management Volume Control Media Device Control Audio Streaming (Listening Broadcast) Note: This functional logic resembles Xiao Ai (Xiaomi’s AI assistant).
-
MCP (Multi-Channel Protocol) (Orange Branch) IoT & Smart Home Control IoT Device Management Smart Home Control MCP Control Protocol Device Status Monitoring Scene Mode Control Information Service Features Weather Information Inquiry News Broadcasting Function Calendar Management Stock Information Inquiry Transportation Information Other MCP Tool Calling Capabilities
-
Memory & Emotion Engine (Purple Branch) Advanced Memory & Learning System Graph Database Long-Term Memory Memory Construction & Integration Memory Recall Mechanism Memory Compression & Simplification Expressive Learning Dynamic Personality Adaptation Realistic Expression Capabilities Chinese Text Generation Emotion System Keyword Response System Reply Frequency Intelligent Control Message Filtering System Advanced Dialogue Functions Focus Chat Mode Thought Chain Display Self-Recognition Processing Multi-turn Contextual Dialogue Intent Recognition Online "Rage Mode"
-
Mobile Support (Blue Branch) Currently, only OLV-PYQT is planned for mobile support.
Summary: This mind map outlines a comprehensive feature roadmap for Open-LLM-VTuber, focusing on four main areas:
- Extensibility via plugins and integrations.
- Smart home and information services through MCP.
- Emotional intelligence and memory systems for realistic interaction.
- Mobile compatibility, currently limited to OLV-PYQT.
@mastwet Thanks!
Regarding the question : Which features should be plugin-ized, and which should remain part of the core system? I think we should separate the features in this way:
Is this feature useful in the majority of use cases? In that case, it should be integrated into the core system. For example, I think Graph Database Long-Term Memory and Emotion System fall into this category.
Is this feature useful for a specific use case? In that case, it should be made into a plugin. For example, I think Discord/Telegram Integration and Calendar Management fall into this category.
@theneur0mancer indeed.However, in my vision, there may be more than one way to implement long-term memory and emotional systems. From this perspective, comprehensive pluginization might be important. Our core development team has even previously discussed making all existing functionalities, including ASR, TTS, and LLM, fully plugin-based. This is also worth exploring, but it would require significant architectural changes. I believe what you mentioned—minimal integration for the emotional system or RAG system—would not only avoid excessive modifications but also allow users to minimize configuration, enabling plug-and-play usage.