SlimeVR
Properly support positional constraints in skeletal model
(aka: "Let me mix slimevr knees with vive tracker hips/feet damnit")
@Louka3000 already got elbows working (:heart:) by using the controllers as a new "root", so they're in a working state. Some more enhanced calibration might be nice for it, but as long as you don't need shoulder tracking (looking at VRChat) it seems to work nicely as-is without touching calibration at all (from my limited experience using joycons)
The main thing currently is that vive trackers don't integrate well into the main skeleton. They apparently don't use their position, only maintain their rotation, and somehow affect the rotation of child trackers? (is there some code specifically for vive trackers already that needs changing?)
Short term, if we can fix whatever weird thing is going on with rotation, override the position of the hip/waist using the vive tracker's actual position, then add parameters to offset the tracker into the right place (especially for people using a tracker on their side, instead of in front) then we'll be gold. I don't know much about server internals right now, but this should require minimal touching of the internal skeleton.
An interesting option is to have the option of performing a repeat of elbow tracking -- use foot trackers as a new root, then do the knees based off of them. BVH is a big concern here, though.
I have some thoughts about root bones in general that I've mentioned in discord, but a vauge idea that might require redoing the way the skeleton works is not worth mentioning in this particular issue. tbh, I'm mostly transcribing a discussion from discord into somewhere perhaps slightly more permenant/visible/referencable.
one thing that would be helpful is a brief explanation of how it works currently. @Louka3000 could you elaborate if you have some time?
As an aside, positional constraints were one of the motivating factors with the implementation of the rust skeletal model. Feel free to reference that preliminary work if its helpful. However, the approach taken there is more advanced than probably what would be used in the java model.
If you are curious, I've taken extensive effort to document the algorithm in plain English here
one thing that would be helpful is a brief explanation of how it works currently. @Louka3000 could you elaborate if you have some time?
Only the rotation of VRTrackers (e.g: Vive trackers) is used. Not the position. They are thus treated the same way as normal SlimeVR trackers (IMUTrackers) minus the reset/mounting.
-
The first step to make this better is to make reset work, which should be very easy to do. (I would however wait for https://github.com/SlimeVR/SlimeVR-Server/pull/251 to be merged!)
-
The second step is to have some form of positional constraint work, but that will require a partial refactor of the HumanSkeleton class at the very least.
As a side note: this whole issue is something I would like to work on (and I'm probably the best person since I got Vive + SlimeVR, and I'm very familiar with how the skeleton works). If you already wrote some code for a positional constraint system in Rust, I could try my hand at porting it in Java. If not, I'll try making it work myself from scratch.
There already is some code written yes, and the algorithm is in english and not rust, if that helps.
The TLDR is:
- The skeleton uses a graph data structure with edges and nodes as distinct objects, with different data stored in each.
- There are input&output positions&rotations
- the solver takes the inputs and computes any missing outputs using FK. Positional and rotational constraints are just copied from the input to the output, no need to solve for those.
- When doing the FK, it doesn't just start at the head and work its way down. It instead starts at any nodes with a known position and works "outwards" (i.e. breadth first traversal). This ensures that position is based on the closest node.
Yes, I had already read your whole documentation when you first sent it to me :p Doing this will require refactoring, but I think the hardest part would be dealing with the tracker imputing and averaging (or just need a lot of ifs...)
I was under the impression that #1222 (aka skeleton constraints) only handled rotational constraints, and not positional constraints? I see it adds a "Use Position" setting, but it doesn't seem to be hooked up to anything?
@Eirenliel @Stermere
Rotational constraints and positional constraints were originally one feature but positional constraints are much harder to get working perfectly. The feature got split into two but the solarXR schema was left unchanged that's why usePosition is a toggle. The positional constraints PR needs some love and I haven't had time to work on it lately, I'm not sure if the bugs in the current implementation are bugs or just limitations of the IK algorithm.
