VRMConverterForVRChat icon indicating copy to clipboard operation
VRMConverterForVRChat copied to clipboard

Viseme issue when ''convert for VRchat''

Open SugarCrashhhh opened this issue 11 months ago • 3 comments

I have an issue when converting for VRchat Fcl_MTH_U and O look fine, but then in the visemes vrc.v_ou and oh look exactly the same and they do not quite look like an ''oh'' nor ''ou''

Pictures of the issue: Unity_V9Tz2rTktv Unity_qVATJiD4W8

Pictures of Fcl_MTH_O and U: Unity_YNJ5yU3np7 Unity_vJ8HIdE1sX

I am using unity 2022 ver installed by the VCC, and running latest version of VRMConverterForVRChat

Is there a way to fix this? Thanks!

SugarCrashhhh avatar Mar 12 '24 10:03 SugarCrashhhh

To generate viseme for VRChat, this extension uses the blend value of Blender add-on “Cats Blender Plugin” and only A, I, and O are used.

https://store.vket.com/items/122 https://github.com/absolute-quantum/cats-blender-plugin/blob/0.13.3/tools/viseme.py

esperecyan avatar Mar 13 '24 00:03 esperecyan

I understand now that it works like Cats Blender Plugin, but this isn't right is it?

The vrc.v_ou and vrc.v_oh are the same, as you can see in the first two pictures So when you speak in VRchat, making an ''o'' or ''u'' sound looks awkward because of this

Could it be an issue with the newer Cats plugin or VRoid studio models? I used CATS plugin in blender before and the visemes looked right, now any model that goes through cats or VRMConverter has this issue

Do you get the same results when using the VRMConverter or Cats plugin?

Thank you

SugarCrashhhh avatar Mar 13 '24 11:03 SugarCrashhhh

Yes, the algorithm has not changed for a long time and neither has the Cats Blender Plugin for a long time.

Although the algorithms for oh and ou are different, they may look the same depending on the Viseme from which they are converted. See below for the actual algorithm. https://github.com/absolute-quantum/cats-blender-plugin/blob/0.19.0/tools/viseme.py#L149-L159

There is no ability to customize this algorithm in this extension, but you can add a callback function that does the post-processing after the conversion is complete.

esperecyan avatar Mar 13 '24 12:03 esperecyan