VTuber_Unity icon indicating copy to clipboard operation
VTuber_Unity copied to clipboard

Model moves erratically when connected with python

Open astroty opened this issue 4 years ago • 5 comments

ezgif-6-4c079af072f6 When connected to the demo.py through --connect (gpu or cpu) the model starts moving her head uncontrollably. In the presented gif it's moving slower than it should be due to recording lag, so it's worse than shown

Hardware used is an i7 5960x with a GTX 980

astroty avatar Oct 01 '20 06:10 astroty

same thing xP

Ubuntu 20.10

ideeej avatar Jan 15 '21 14:01 ideeej

Me too

Windows 10, RTX 2070, i9-10900

MainKronos avatar Feb 05 '21 13:02 MainKronos

Jup same here with --cpu mode

Arch (Kernel 5.10.16-arch1-1), Ryzen 5 2600

bytedream avatar Feb 19 '21 15:02 bytedream

Same here in --cpu mode

Windows 10, Intel Core i3-6098P

Spanish Language

Lanzet avatar Mar 07 '21 23:03 Lanzet

I think I found the culprit for this. On the debugging window from demo.py the whole alignment to my face always looked fine. So I downloaded the unitychanproject.zip und dug around there. In the file UnityChanControlScriptWithRgidBody.cs we recieve the data that demo.py sends. The culprit there is the float.Parse() function which might work different depending if you're system is configured that your decimal separator is a comma "," or a point "." So the relatively small values with the wrong decimal separator seem to get interpreted as bigger than they actually are because it just seem to skip the decimal separator that isn't defined as the current default and ignore any leading zeros as well. So the strings "-11.4411" and "0.0548" suddenly become -114411 and 548 instead.

the lines 72-77 where float.Parse() appears:

roll = float.Parse(res[0])*0.4f + roll*0.6f;
pitch = float.Parse(res[1])*0.4f + pitch*0.6f;
yaw = float.Parse(res[2])*0.4f + yaw*0.6f;
min_ear = float.Parse(res[3]);
mar = float.Parse(res[4])*0.4f + mar*0.6f;
mdst = float.Parse(res[5]);

To fix that I just also gave float.Parse the "CultureInfo.InvariantCulture" argument so it is forced to interpret points as the decimal separator when your default culcture (in my case German) uses another decimal seperator

fixed lines 72-77 of UnityChanControlScriptWithRgidBody.cs:

roll = float.Parse(res[0], CultureInfo.InvariantCulture)*0.4f + roll*0.6f;
pitch = float.Parse(res[1], CultureInfo.InvariantCulture)*0.4f + pitch*0.6f;
yaw = float.Parse(res[2], CultureInfo.InvariantCulture)*0.4f + yaw*0.6f;
min_ear = float.Parse(res[3], CultureInfo.InvariantCulture);
mar = float.Parse(res[4], CultureInfo.InvariantCulture)*0.4f + mar*0.6f;
mdst = float.Parse(res[5], CultureInfo.InvariantCulture);

This also needs another dependency:

using System.Globalization;

Haven't checked yet if this fix will create problems if the default culture uses a point as their decimal separator. At least this fixes it for me. Sadly this part of the project isn't actually in this repository or in it's own repository. (if it is I couldn't find it).

Balvald avatar May 13 '21 12:05 Balvald