Investigate mismatches between SKEL model distribution in nimblephysics and OpenSim

[nickbianco]

trafficstars

Issue reported here: https://simtk.org/plugins/phpBB/viewtopicPhpbb.php?f=91&t=17422&p=48432

[nickbianco]

Related, there seems to be an issue with visualizing an EllipsoidJoint in the GUI:

In the Simbody visualizer I get this:

The latter image seems to confirm that there is indeed a mismatch between the nimblephysics and OpenSim EllipsoidJoint definitions.

[nickbianco]

FYI @aymanhab, there might be a GUI issue related to EllipsoidJoint based on my comment above.

[AlbertoCasasOrtiz]

This is the output returned when trying to load the model with the big blue sphere in opensim 4.4:

Object::newInstanceOfType(): object type '{ConstantCurvatureJoint}' is not a registered Object! It will be ignored.

[nickbianco]

The ConstantCurvatureJoint wasn't added until 4.5, so that error is expected. We should see if loading a model with an EllipsoidJoint and no ConstantCurvatureJoint works in 4.4.

[aymanhab]

Thanks @nickbianco Can you attach the model or a link to it here so I can try the 4.5 GUI?

[nickbianco]

@aymanhab here is the OpenSim version of Marilyn Keller's SKEL model, with markers removed since some of those were incorrect.

bsm_no_markers.zip

[aymanhab]

For the record, the problematic big blueberry sphere is coming from scapulothoracic_l joint which happens to have -ve radii (-.08, -.19999, -.083)

[nickbianco]

Aha! Thanks for finding that @aymanhab.

[nickbianco]

Another update, I got the markers to visualize after fixing the frame socket names. These also look wrong:

[nickbianco]

One more: loading the sample motion has the model's spine at nearly a right angle:

This is definitely an issue with the ConstantCurvatureJoint definition, since the motion is accompanied by the following output to the console:

WARNING! ConstantCurvatureJoint position outside of its supported range! X rotation reached position 6.35505, which is above upper bound 1.56079. This will lead to unphysical behavior. Please adjust your model or simulation to avoid this state.
WARNING! ConstantCurvatureJoint position outside of its supported range! Y rotation reached position 6.29097, which is above upper bound 1.56079. This will lead to unphysical behavior. Please adjust your model or simulation to avoid this state.
WARNING! ConstantCurvatureJoint position outside of its supported range! X rotation reached position 6.35505, which is above upper bound 1.56079. This will lead to unphysical behavior. Please adjust your model or simulation to avoid this state.
WARNING! ConstantCurvatureJoint position outside of its supported range! Y rotation reached position 6.29097, which is above upper bound 1.56079. This will lead to unphysical behavior. Please adjust your model or simulation to avoid this state.
WARNING! ConstantCurvatureJoint position outside of its supported range! X rotation reached position 6.35505, which is above upper bound 1.56079. This will lead to unphysical behavior. Please adjust your model or simulation to avoid this state.
WARNING! ConstantCurvatureJoint position outside of its supported range! Y rotation reached position 6.29097, which is above upper bound 1.56079. This will lead to unphysical behavior. Please adjust your model or simulation to avoid this state.
WARNING! ConstantCurvatureJoint position outside of its supported range! X rotation reached position 6.35505, which is above upper bound 1.56079. This will lead to unphysical behavior. Please adjust your model or simulation to avoid this state.
WARNING! ConstantCurvatureJoint position outside of its supported range! Y rotation reached position 6.29097, which is above upper bound 1.56079. This will lead to unphysical behavior. Please adjust your model or simulation to avoid this state.
WARNING! ConstantCurvatureJoint position outside of its supported range! Y rotation reached position -6.36803, which is below lower bound -1.56079. This will lead to unphysical behavior. Please adjust your model or simulation to avoid this state.
WARNING! ConstantCurvatureJoint position outside of its supported range! Y rotation reached position -6.36803, which is below lower bound -1.56079. This will lead to unphysical behavior. Please adjust your model or simulation to avoid this state.
WARNING! ConstantCurvatureJoint position outside of its supported range! Y rotation reached position -6.36803, which is below lower bound -1.56079. This will lead to unphysical behavior. Please adjust your model or simulation to avoid this state.
WARNING! ConstantCurvatureJoint position outside of its supported range! Y rotation reached position -6.36803, which is below lower bound -1.56079. This will lead to unphysical behavior. Please adjust your model or simulation to avoid this state.

[nickbianco]

After corresponding with @MarilynKeller, she informed me that the missing geometry files were not originally created from existing VTP files. After attempting to convert the distributed files (via this script) and loading them into OpenSim, she received an XML parsing error:

Visualizer couldn't open lunate_l.vtp because:
SimTK Exception thrown at PolygonalMesh.cpp:411: Error detected by Simbody method PolygonalMesh::loadVtpFile(): Attempt to load a VTK PolyData (.vtp) file from file name '/Users/mkeller2/Dropbox/MPI/TML/osim_models/test_vtp/Geometry/lunate_l.vtp' failed with message: SimTK Exception thrown at Xml.cpp:723: Error detected by Simbody method Xml::element::getValue(): Element <DataArray> is not a value element. (Required condition 'isValueElement()' was not met.)

Attached below is an example of a converted file.

lunate_l.zip

[aymanhab]

Here's the vtp file converted to .obj file which should work out of the box by changing the name in the osim file to lunate_l.obj lunate_l.zip

[nickbianco]

Nice! I used @MarilynKeller's script to convert the other PLY files into VTP files (see attached). @aymanhab could you convert those to obj files too?

Geometry.zip

[aymanhab]

Sure, will write a script to do it and will post it here by tomorrow.

[MarilynKeller]

Can OpenSim load .obj meshes ? I assumed it was only .vtp. That would make things much easier.

[aymanhab]

Absolutely, we support vtp, obj and stl files for meshes.

[aymanhab]

Geometry_obj.zip

[nickbianco]

Thanks @aymanhab! These worked great:

Note that the geometries for both patellas were not included in the BSM distribution Geometry folder, so I'm using the VTP files from the GUI distribution for those.

[nickbianco]

It's clear that the EllipsoidJoint definitions are wrong, and the ConstantCurvatureJoint coordinates from the motion file still produce a ~90 degree lean in the torso. I'll now start looking to the mismatches in these joint definitions.

@MarilynKeller if you have any details on how the model was exported from nimblephysics to OpenSim, that could be useful here.

[MarilynKeller]

As far as I know, the model is not really "exported" from nimblephysics. It was created as a .osim xml, and parsed by nimblephysics/dart directly here
The constant curvature joint is defined here and the ellipsoid joint here.

I hope this helps.

[nickbianco]

That is helpful to know. This suggests that our test coverage of these joints in OpenSim isn't as quite as strong as it could be.

I will continue to investigate.

[davidpagnon]

I need to choose a full-body OpenSim model, and I have potential applications with SMPL meshes, so the BSM model from SKEL would be the perfect fit. I'm not sure how to help but if I can, I will!

What would be the idea? Editing the joint definitions in Nimble Physics, or in OpenSim? Either way, the model is not going to work in one of these applications anymore, right?

[nickbianco]

@davidpagnon the first step would be to identify the differences in the implementation of the EllipsoidJoint in nimblephysics and OpenSim. The nimblephysics implementation is here, and the Simbody implementation (which OpenSim uses) is here. The implementation of OpenSim::EllipsoidJoint uses Simbody's Ellipsoid mobilized body class.

EDIT: I corrected the link for Simbody's implementation of Ellipsoid.

[davidpagnon]

Thanks! I won't be able to work on it within the next two weeks, but if it has not been solved by then I'll have a look!

[nickbianco]

Sounds good @davidpagnon! Thanks for being willing to take a look.

[davidpagnon]

Hi all, unfortunately, I've been busier than I expected, and I can't give a date when I'll be able to dive into it... If anyone is on it, I'm interested in any case! I'll keep you updated if my schedule clears up.

[nickbianco]

Hi @davidpagnon, thanks for the update and no worries. Actually, @keenon and I recently realized that the ellipsoid joint between Nimble and OpenSim are implemented very differently, so there's not much to investigate here. We essentially need to rewrite the Nimble ellipsoid joint to match OpenSim, which is some nontrivial geometry work.

[davidpagnon]

Thanks, that's what I figured, and I got overwhelmed 😅 Not to put you under pressure or anything, but do you have a rough idea of when it will be done (and when it will be supported by Addbiomechanics )?

[nickbianco]

We currently don't have a timeline for this fix. But we will reprioritize as more people find it useful.

@keenon, I'm wondering if the easiest solution is just to implement your version of the ellipsoid joint in OpenSim. OpenSim's EllipsoidJoint relies on Simbody's notion of a mobilizer, which I'm not sure dart/Nimble actually support.

[Arish2022]

Hi @nickbianco,

Thank you for your work on the model. I noticed a few missing elements: the left and right patella are listed in the osim file (<mesh_file>l_patella.obj</mesh_file>, <mesh_file>r_patella.obj</mesh_file>, but their geometries are not included. I've attached screenshots for reference. Additionally, the clavicles are absent from the osim file.

Do you have any thoughts on how to address these issues?