agents Stream Manipulation using Custom Plugins

Due to lack of Documentation in plugins section, I'm not sure how to build one let alone how to customise the already existing one. I want manipulate the input and output stream using the custom plugins as descirbed in the documentation but don't know how to do so. It just says:

The plugin framework is designed to be extensible, allowing anyone to build their own plugin. Your plugin can integrate with various providers or directly load models for local inference.

By adopting the standard STT or TTS interfaces, you can abstract away implementation specifics and simplify switching between different providers in your agent code. https://docs.livekit.io/agents/plugins/#Building-your-own

Jun 19 '24 05:06 parshvadaftari

I also don't know, but look at these plugins and try to adapt it / rewrite it? https://github.com/livekit/agents/tree/main/livekit-plugins

Jul 18 '24 12:07 ChrisFeldmeier

Hey @parshvadaftari - custom behavior can be achieved by subclassing one of our base classes (STT, TTS, or LLM). For example, the openai LLM is implemented in this file: https://github.com/livekit/agents/blob/main/livekit-plugins/livekit-plugins-openai/livekit/plugins/openai/llm.py

Which is a subclass of this base class: https://github.com/livekit/agents/blob/main/livekit-agents/livekit/agents/llm/llm.py

It sounds like you aren't necessarily trying to create a brand new plugin, but rather a wrapper plugin that can "map" the output of an existing plugin. Is that correct?

Jul 29 '24 22:07 keepingitneil

Hey @keepingitneil , I tried making my own plugin, but failed and used a different approach of making changes in the https://github.com/livekit/agents/blob/main/livekit-plugins/livekit-plugins-openai/livekit/plugins/openai/llm.py file and manipulating the output using my requirements.

Is there any simply way to do the same? Like you mentioned about the custom plugin, any code snippet or any example of it would be helpfull, example which is not on the website.

Jul 31 '24 04:07 parshvadaftari

I closed the issue by mistake, apologies for it.

Jul 31 '24 05:07 parshvadaftari

Hey @parshvadaftari , did you find any documentation or snippet of code ?

I try to do a pluging so we could use any model.