How to add equipment or handheld objects

[loreofblasius]

trafficstars

Great work ! In your trailer you show an OpenSim skeleton riding a bicycle. How do you integrate a handheld device in the model. Do you extend the OpenSim model with additional "bones" and "joints", or what would you recommend ?

[davidpagnon]

Hi, thanks for you interest!

Here is what I did:

1. Human being
   • regular pipeline to obtain the trc file
   • scale the model of the person
2. Bike
   • Trained a DeepLabCut model
   • Converted the results to OpenPose json format
   • Edited the CUSTOM model in Config.toml, in order to triangulate the DeepLabCut points
   • Created an OpenSim model. I did it the old school way by creating stl files (Blender works, and so do many other pieces of softwares: search "create stl file" on Google) and then creating an .osim file with joints, bodies, and markers by hand. But you should check https://www.opensimcreator.com/, I have not tried it but it might be much more straightforward. Note that you don't have to have good-looking bodies, you could create a sphere or anything for each of them as long as their properties are right
   • Scale the model you created from DeepLabCut. You will have to edit the Scaling_setup.xml file so that it takes the right keypoints.
3. From that, you have two options :
   • perform IK independently for both models
   • Or if you want both person and deeplabcut models to be constrained to each other, you can assemble the trc files with this script: trc_combine.py, and then perform IK on both models together

More information here: https://github.com/perfanalytics/pose2sim?tab=readme-ov-file#with-deeplabcut

[davidpagnon]

This issue has been marked as stale. Please feel free to reopen it if needed!