overte-org
Add support for OVRLipSync-style visemes
We lack viseme support, but unfortunately there doesn't seem to be any proper library for it.
The industry standard is OVRLipSync which is closed source, proprietary, and doesn't support Linux. Since lip-syncing is handled client side, we could theoretically make it available as an optional plugin, but probably no one would want to work on it, since it wouldn't work on Linux without depending on Wine or Qemu/Binfmt. Alternatively, there is a well documented open source implementation for Unity which we could use as a starting point to create a C++ library. https://github.com/hecomi/uLipSync
Since almost all social VR platforms are in the same boat here, maybe it could be viable to collaborate on such library. Neos seems to have issues with OVRLipSync, so if a Neos 2 ever emerges, they could be a candidate to collaborate with. V-Sekai for instance is going to have exactly the same problem as we do. As an open-source platform, they cannot just include such proprietary library.
There is apparently also some open source neural network based stuff made by Chinese anime girls: https://github.com/Magicboomliu/Viseme-Classification https://github.com/marty1885/OpenViseme
To directly give an update about the issue on Resonite side if you don't want to check the issue
The community is working on https://github.com/KyuubiYoru/OpenLipSync this is still experimental, but works (tho lack laughter detection compared to ovrlipsync)
A C# implementation can be found here https://github.com/KyuubiYoru/VisemesAtHome (it's a mod to implement it in Resonite)
No idea if this'll be usable for y'all, but it's here
@Naia-love thank you for sharing this. If it ends up working okay it indeed looks like a viable option.
Resonite have implemented it directly https://youtu.be/oSVugEX_4ng / https://store.steampowered.com/news/app/2519830/view/526480495508196129?l=english
using it by default if OVRLipSync isnt available, leaving it as an option otherwise
