drake icon indicating copy to clipboard operation
drake copied to clipboard

Consider storing sdformats and urdfs adjacent to external meshes

Open jwnimmer-tri opened this issue 5 years ago • 2 comments

See https://drake.mit.edu/model_version_control.html for background.

We are storing models' meshes, textures, and other large files in https://github.com/RobotLocomotion/models, so that a git clone of Drake does not grow inordinately large. In the procedures for that, we say:

Please do not commit files that are generally small, like *.sdf or *.urdf files, in RobotLocomotion/models; instead, please commit those directly.

The thought was that these small text files might change or fork frequently (e.g., to change collision models, or other contact properties), and so laundering those changes through an external repository would be cumbersome. Furthermore, within Drake we have CI that can validate that the files are well-formed, can be loaded by our parser, etc. If we stored the text files along with the meshes, we would need to modify those procedures.

On the other hand, it is also awkward to add new models, or preview the current models, with the files split between git repositories.

Relatedly, it's not unusual for us to inherit the models from a third-party repository, which means we must propagate the licensing information. Pushing it to two places is awkward, and having third-party code (e.g., urdf) committed into drake in the manipulation/models folder (instead of third_party) makes license sanity checking for distribution and release more awkward.

We should consider changing the policy, and moving the sdformat and urdf files into the external repository, in cases where they have large files associated with them.

jwnimmer-tri avatar Aug 27 '20 14:08 jwnimmer-tri

Note that this might have implications for #10531. It might mean that for things like iiwa_description, we should have its own package.xml file and not bake it into the all-drake package. Even if we call it drake_iiwa_description to set it apart from upstream.

jwnimmer-tri avatar Oct 01 '20 19:10 jwnimmer-tri

Update from discussing this with Russ:

In solving this issue, now that we've decided on a single package.xml for Drake, we'll need to decide what to do about package.xml file(s) for the RobotLocomotion/models repository.

If we want that repository to stand on its own (and that seems like a nice goal), then we should add one or more package.xml files within RobotLocomotion/models. Once we do that (and when the files leave Drake), should those package.xml file(s) be added into the PackageMap default constructor automatically?

We'll need to iterate on this design a little bit.

jwnimmer-tri avatar Jul 06 '22 18:07 jwnimmer-tri

... we should add one or more package.xml files within RobotLocomotion/models.

Done in https://github.com/RobotLocomotion/models/pull/22.

Once we do that (and when the files leave Drake), should those package.xml file(s) be added into the PackageMap default constructor automatically?

Yes. Done in https://github.com/RobotLocomotion/drake/pull/18919.

What remains here is just the boring work of copying the SDFormat files (not moving!) and deprecating their current package://drake paths, and the less-boring work of solving how and when to run xacro and linters on the files in the other repository.

jwnimmer-tri avatar Apr 12 '23 15:04 jwnimmer-tri

... how and when to run ... linters on the files in the other repository.

#19986 proposes that Drake CI will run the linters. That means a PR to drake_models will not have CI coverage until the twin Drake PR is opened up.

jwnimmer-tri avatar Aug 14 '23 03:08 jwnimmer-tri

In order to close #11913, I'll add one more specific task here: the files at examples/pr2/models/pr2_description should move into drake_models.

jwnimmer-tri avatar Mar 19 '24 22:03 jwnimmer-tri

FYI I'm working on this now. Getting all of the model files in one place is pretty much a requirement for converting them to glTF without headaches.

jwnimmer-tri avatar Mar 25 '24 21:03 jwnimmer-tri