t41372
Tool use (function calling)
i wish i could add custom responces so if i say turn office lights on it says/dos somthing speciffic like saying okay turning the light on and doing -X POST https://xxx.xxx.x.x/api/webhook/turn_office_lights_off
See #125 We need a function calling interface. Maybe we will add it in v1.2? (v1.1 is about to release)
now that I have you here I just wanted to take a moment to express my appreciation for this incredible project and its capabilities. The impact it has already made is truly remarkable.
I work in the care sector with individuals who have intellectual disabilities—people with young minds in older bodies. They absolutely love interacting with the VTuber, engaging with it throughout the day. Thanks to you and your team, about 24 individuals who were struggling with loneliness and depression now have someone to interact with, bringing them joy and companionship.
The potential of this project is immense, and I truly believe it has the power to transform the care sector in profound ways. On behalf of our institution, I want to sincerely thank you for making such a difference in their lives.
Ps I also use it privately and wish for twitch chat integration but that's not as important as the command capability
It's heartwarming to know that. I never imagined this project would have such an impact on elderly care. Anyway, thank you for sharing this story, we will continue to work to make the project better.
btw The number one request from our clients is to hear her sing, but I know making that work would be a huge challenge. So, I told them she has a sore throat due to a cold—now they want to give her cold medicine! i have no idea how to win this discussion with them xD
Sing basically is another function call. I tried implementing this in an old fork (using tags to trigger songs), there are some challenges:
- Original songs need to be prepared in advance, and we should match the song (sometimes people don't say the song name, maybe they say the singer or something else)
- Human voice extraction + voice conversion is needed (if you want to adapt to the character's voice), see RVC. We should avoid requiring users to do it manually
I'm thinking we'll tackle the singing feature once we get the function call finished.
MCP server has been implemented in the dev branch and will be released as a part of 1.2. This issue will be closed once 1.2 is released.
