Create a specification for primitive physics shapes (boxes, spheres, etc)

[aaronfranke]

EDIT: Please note that the specific details in this proposal no longer reflect what I believe to be the best approach. The goals behind this proposal, specifying geometric shapes, are still valid.

I'm working on Godot Engine and I would like the ability to specify primitive geometric shapes in a GLTF file for the purpose of being read and used by a physics system and by other systems such as audio.

I have seen that #1135 exists, and it has received some negative feedback. It's a huge proposal which includes things like physics materials, joint constraints, gravity, and more. Really the full extent of that proposal is overkill for what I want, so I am opening this proposal to hopefully get something simpler added to the specification.

Godot has a GLTF exporter, so I will try to be as specific as possible as to what the output currently looks like in the .gltf file and what the desired output would be if this specification was added.

Here is a screenshot of an example scene I am working with (core.tscn from the TPS demo):

And here is that same screenshot but with the meshes hidden, so that only the primitive colliders are visible:

When exporting, the collision primitives easily visible in the second screenshot are not preserved. However, each of the collision shapes do get exported with a name, rotation, and translation, just with no shape defined. It looks like this in the .gltf:

    {
      "extensions": {},
      "name": "MainFloor2",
      "rotation": [0, -0.078459, 0, 0.996917],
      "translation": [21.6304, -14.5, 3.42592]
    },

If allowed by the spec, the information about primitive shapes could look something like this:

    {
      "extensions": {
        "primitive_box": { "size": [11, 4.2, 4] }
      },
      "name": "MainFloor2",
      "rotation": [0, -0.078459, 0, 0.996917],
      "translation": [21.6304, -14.5, 3.42592]
    },

Here is an overview of what the proposed API could look like. All numbers must be >= 0.

• Box
  • Suggested name: primitive_box
  • Has size member of type Vector3 (three numbers), which specify the "diameter" of the box, so size of [1, 1, 1] means a 1x1x1 cube.
    • Alternative implementation: Replace size with extents that defines the "radius" of the box, so extents of [1, 1, 1] means a 2x2x2 cube.
• Sphere
  • Suggested name: primitive_sphere
  • Has radius member which is one number that defines the distance from the center to the surface.
• Capsule
  • Suggested name: primitive_capsule
  • Has radius member which is one number that defines the distance from the center to the surface of both "cap" semi-spheres.
  • Has height member which is one number that defines the distance from the very bottom to the very top.
• Cylinder
  • Suggested name: primitive_cylinder
  • Has radius member which is one number that defines the distance from the center to the edge of both circles.
  • Has height member which is one number that defines the distance between the centers of both circles.
  • Note: Not all physics systems and other systems support cylinders, because doing so is more complex than the other primitives. In this case, a convex hull will need to be used.

Arguments can be made for a few other primitive types, such as cones, cylinders with different end sizes, and rays. However, the above are the only ones I'm familiar with and I know to be broadly useful.

This information would be much more efficient than storing mesh triangles for each physics object, and would make it easier to move scenes between projects and even between different game engines. These shapes could be used by more than just collision, for two examples in Godot, Area nodes can be used to change the audio settings for some part(s) of the level, and can also be used to change the gravity for some part(s) of the level.

---

This is really two proposals in one, with the below dependent upon, but really separate from, the above. Without the below, software would just have to guess as to what the purpose of the shapes are, or let the user specify with import settings.

To make this proposal even more useful, it would be nice to be able to define the purpose of the shapes. Because physics bodies are very often composed out of multiple shapes, each object with a shape would be children of a parent physics body of some kind. In the case above, one of them looks like this in the .gltf file:

    {
      "children": [
        259, 260, 261, 262, 263, 264, 265, 266, 267, 268, 269, 270, 271, 272,
        273, 274, 275, 276, 277, 278, 279, 280, 281, 282, 283, 284, 285, 286,
        287, 288, 289, 290, 291, 292, 293, 294, 295, 296, 297, 298, 299, 300,
        301, 302, 303, 304, 305, 306, 307, 308, 309, 310, 311, 312, 313, 314,
        315, 316, 317, 318, 319, 320, 321, 322, 323, 324, 325, 326, 327, 328,
        329, 330, 331, 332, 333, 334, 335, 336
      ],
      "extensions": {},
      "name": "RadialColliders2",
      "rotation": [0, 0.782608, 0, 0.622515]
    },

Then, the parent body would have its purposed defined, something like this:

    {
      "children": [
        259, 260, 261, 262, 263, 264, 265, 266, 267, 268, 269, 270, 271, 272,
        273, 274, 275, 276, 277, 278, 279, 280, 281, 282, 283, 284, 285, 286,
        287, 288, 289, 290, 291, 292, 293, 294, 295, 296, 297, 298, 299, 300,
        301, 302, 303, 304, 305, 306, 307, 308, 309, 310, 311, 312, 313, 314,
        315, 316, 317, 318, 319, 320, 321, 322, 323, 324, 325, 326, 327, 328,
        329, 330, 331, 332, 333, 334, 335, 336
      ],
      "extensions": {
        "shape_usage": "static_collision"
      },
      "name": "RadialColliders2",
      "rotation": [0, 0.782608, 0, 0.622515]
    },

The exact API of this part is something I'm less sure about than the above shape API. We want to be able to specify many different types of physics bodies to provide as much information as possible to software, including rigid, kinematic, static, character, area detection, audio manipulation, and more. The specific details of these things (like defining mass or whatever) are simple and specific enough that we should consider letting each piece of software handle it on their own. It could be argued that these things (like mass etc) are worth defining in GLTF, but not in this proposal specifically. In this proposal the goal is to be able to define primitive geometric shapes, and likely also define the purpose of the shapes.

[Leadwerks]

I have written a draft here of a spec that handles collision shapes: https://github.com/Leadwerks/glTFExtensions/blob/main/ULTRA_collision_shape/README.md

It covers the main features of Blender, Newton Dynamics, and Bullet Physics. Convex hulls are explicitly defined, and stored in binary. Polygon mesh shapes support a varying number of indices per face, or can use a constant number of indices (i.e. triangle or quad meshes).

Multiple shapes can be defined per node. I have never seen a hierarchy of collision subobjects in a physics lib, we always just use a group of subobjects that are treated as one.

The spec does not include physics properties or joints, as those can be implemented in a separate self-contained extension if desired.

[javagl]

@Leadwerks There is an open PR that seems to have a similar (or maybe even the exact same) goal: https://github.com/KhronosGroup/glTF/pull/2087 (Disclaimer: I have not read into the details of your draft spec (beyond noticing a typo in coliisionShapes) or the PR in all detail, but from quickly skimming over them, they seem to be very similar). Maybe you want to share your thoughts and feedback in that PR - also to establish a "link" and identify commonalities or differences between the proposals.

EDIT: Sorry, that may not have been the perfect fit. The other PR seems to focus on using the glTF built-in structures to define the collision meshes, whereas your proposal might be seen as a combination of ~"allowing defining certain primitives (box, sphere...)" and "allowing to use something (namely a mesh or a one of these primitives) for collision detection". One overlap might be the "convex hull" definition, though...

[aaronfranke]

@javagl Indeed, these are related, but not the same. This proposal is about defining shapes, not meshes. While you can make a cube out of 8 vertices and 12 triangles, it's not nearly as efficient for physics engines to handle compated to just defining a box collider. Both collision shapes and collision meshes can be used for physics and more, so they are related in how they are used.

[javagl]

@aaronfranke Yes, sorry, I'll have to read the PRs/proposals more closely (i.e. this one, the one that Leadwerks mentioned, and the one from MAXAR that I linked to). From quickly skimming over them, I can only make some vague, high level points (that may or may not be obvious):

1. Having the possibility to define geometry via "primitives" could be useful.
2. Having the possibility to use "geometry" for collision detection could be useful.

Regarding 1.: The term 'primitive' is a bit loaded, due to mesh.primitives. But here, it dedicatedly and only refers to the primitive geometric shapes that can be described "parametrically", like a cube, sphere, or cylinder. This could include ...

• generalizations: cube -> box, sphere -> ellipsoid, ...
• combinations: "cylinder + 2 half spheres = capsule", ...

Modeling that cleanly could already be difficult. Keeping the use case of "collision detection" in mind: Collision detection with a sphere is trivial. Collision detection with an ellipsoid is pretty much non-trivial compared to that. Also, for pure rendering this should probably include "half-spheres", maybe "cylinder segments", and maybe even slightly more complex objects like a "chamfer box". Or, more abstractly, some of the questions are:

• Which objects can be described "parametrically"?
• What should the exact parameters be? For example, you mentioned a possible ambiguity of capsule.height. One could also ask whether a cylinder should have a numberOfCircularSegments and numberOfLongitudinalSegments, for example...

Regarding 2.: The main point here is: It would be great if it was possible to use geometry for collision detection transparently, regardless of whether it is a "primitive", or a glTF mesh. I think that's what you meant with the shape_usage. So it would be great if someone could write (pseudocode-glTF) :

{
    meshes: {
        /* 0 : a standard glTF mesh of a cube */
        /* 1: a primitive, via "extension { shape: cube } }"
    }
    collisionThings {
        { mesh: 0 }; // Uses the standard glTF mesh (of a cube) for collisions
        { mesh: 1 }; // Uses the primitive cube shape for collisions
    }
}

But I'm pretty sure that some of these aspects are already covered in the existing proposals, to some extent.

---

One thing that I found "surprising" from the first, quick glance at the ULTRA_collision_shape draft: The transform (translation/rotation/scale) was contained in the shape itself. One could consider to just attach that shape to a node, and use the existing node transform for the transformation.

Another thing is the aforementioned overlap between the concepts for a "convex hull". While it could be presented with a glTF mesh, I think that one could consider defining a special primitive for that. It could be defined by a few properties, similar to size and radius for cube and sphere. But here, the properties would be the normals/distances of the planes that define the intersected half-spaces that make up the convex hull:

convexHull {
    planes: [
        { normal: { 0.7, 0.0, 0.7 }, distance: 0.8 },
        { normal: { -0.7, 0.0, 0.7 }, distance: 0.2 },
        { normal: { 0.7, -0.7, 0.0 }, distance: 0.6 },
        ...
    }
}

The reason for that is that for collision detection, it would be important to know that this mesh is guaranteed to be a convex hull. (Checking that manually for an arbitrary glTF mesh would hardly be feasible...)

[Leadwerks]

I'm not in madly in love with any particular spec as long as it contains the most common features that Blender, Newton, Bullet, and PhysX will use. I'm happy to help work on the spec or plugin if we can agree on the design.

Of all the people talking about this, who actually has experience working with the glTF exporter for Blender? That is probably the person who will be calling the shots on this.

In my document, the offset matrix of the shape is meant to be multiplied by the node's matrix. This was designed around the collision offset matrix in Newton Dynamics, but encompasses the "axis" functionality in the Blender physics spec.

[fire]

Since this proposal was written, work on the OMI_collider and OMI_physics_body has progressed. Hoping to see reviews.

[mrmetaverse]

Since this proposal was written, work on the OMI_collider and OMI_physics_body has progressed. Hoping to see reviews.

https://github.com/omigroup/gltf-extensions/pull/63#issuecomment-1148045913

[mikedh]

I like the simple nature of this proposal! I'm not totally sure that the collision details ("has sensor", physics properties, etc) are necessarily the same as "I just want a box in a scene."

I ended up with something close to this proposal when I wanted a spec for the "lowest common denominator" primitive-as-JSON exports, here's the JSONschema. Basically it's the minimum reproducible information (i.e. for a cylinder height and radius), a kind="cylinder" field with a pattern, plus a possibly controversial transform, which is "baked" to move the box or whatever before it's added to a scene graph.

[aaronfranke]

Since opening this proposal, OMI_collider and OMI_physics_body have progressed. OMI_collider now lives in a new pull request here https://github.com/omigroup/gltf-extensions/pull/118, and I have been working on implementing these extensions in Godot here https://github.com/godotengine/godot/pull/69266. Both import and export is supported, so Godot can be used as a tool to add physics information to GLTF files with a graphical editor.

These extensions support all of the use cases I mentioned in the OP, but in a cleaner way. OMI_collider supports primitive shapes, such as boxes and spheres, and also convex hulls and concave meshes. OMI_physics_body supports combining colliders into one body and specifying the purpose of them, such as if it's static or rigid or a trigger.

Here is a test project with several simple test GLTF files: Godot_GLTF_OMI_physics.zip

I'll give some specific responses to the comments in this thread:

@Leadwerks Compared to your ULTRA_collision_shape, OMI physics does not specify position or rotation, which is not necessary information since it's already present on the GLTF nodes that the colliders are attached to. OMI doesn't support cone shapes, and there are no ellipsoids except for spheres. For the mesh-based ones, the concave and convex mesh shapes in OMI just point to one of the meshes in the GLTF "meshes" array.

@erikdahlstrom MAXAR_collision_geometry is a very simple extension, what OMI physics supports is mostly a superset of it. The requirements of this extension are a bit weird, like collision is assigned directly to meshes instead of nodes, and meshes can't use themselves as a collision shape? Also, OMI has OMI_physics_body for specifying physics bodies, but this spec seems focused on just the shapes.

@mikedh Compared to your spec, OMI physics does not specify any transforms, which is not necessary information since it's already present on the GLTF nodes that the colliders are attached to. OMI physics does not have WKT or extrusion shapes, and OMI physics makes the distinction between convex hull and concave mesh shapes. Also, OMI has OMI_physics_body for specifying physics bodies, but this spec seems focused on just the shapes.

@javagl You've made many good points in this thread, thank you! For OMI physics we're keeping things fairly simple, driven by what's commonly needed, so for example we do not have numberOfCircularSegments for cylinders. We're also specifying convex hulls using an array of points, since all other information found in a regular mesh is not useful for convex hulls. Let me know what you think of OMI physics, I think it matches what we have been hoping for.

[javagl]

Let me know what you think of OMI physics,

I'm not deeply enough involved here (on a technical level, and also in view of the engines/implementations that might use or support these extensions) to provide profound, technical feedback. I could look at a particular extension specification on a "high level" (e.g. whether the JSON schema is consistent and in line with other parts of the glTF, and maybe raise a few domain-specific questions). But beyond that, my thoughts have mainly been related to trying to carve out the "description of geometric objects" in a way that can be reused elsewhere (also see my latest comment at https://github.com/eoineoineoin/glTF_Physics/issues/1#issuecomment-1331001917 ). Whether or not that's feasible is hard to say without digging deeper into each proposal.

[erikdahlstrom]

@aaronfranke I'm leaving this for @bjornblissing to answer, as I'm no longer with Maxar.