Integrate gltfpack for mesh and texture compression

[lilleyse]

trafficstars

meshoptimizer is a library for optimizing and compressing models. It includes a tool gltfpack that applies all the optimization routines automatically to a glTF. The two compression extensions that be can chosen from are KHR_mesh_quantization and EXT_meshopt_compression.

gltfpack is provided as an npm package which can be found here: https://www.npmjs.com/package/gltfpack. The package contains a prebuilt WASM binary of gltfpack. It does not provide a JavaScript API so we'll need to call the binary directly and pass over command line arguments.

Here's the list of CLI arguments:

Usage: gltfpack [options] -i input -o output

Basics:
        -i file: input file to process, .obj/.gltf/.glb
        -o file: output file path, .gltf/.glb
        -c: produce compressed gltf/glb files (-cc for higher compression ratio)

Textures:
        -te: embed all textures into main buffer (.bin or .glb)
        -tb: convert all textures to Basis Universal format (with basisu executable); will be removed in the future
        -tc: convert all textures to KTX2 with BasisU supercompression (using basisu executable)
        -tq N: set texture encoding quality (default: 50; N should be between 1 and 100
        -tu: use UASTC when encoding textures (much higher quality and much larger size)

Simplification:
        -si R: simplify meshes to achieve the ratio R (default: 1; R should be between 0 and 1)
        -sa: aggressively simplify to the target ratio disregarding quality

Vertices:
        -vp N: use N-bit quantization for positions (default: 14; N should be between 1 and 16)
        -vt N: use N-bit quantization for texture coordinates (default: 12; N should be between 1 and 16)
        -vn N: use N-bit quantization for normals and tangents (default: 8; N should be between 1 and 16)

Animations:
        -at N: use N-bit quantization for translations (default: 16; N should be between 1 and 24)
        -ar N: use N-bit quantization for rotations (default: 12; N should be between 4 and 16)
        -as N: use N-bit quantization for scale (default: 16; N should be between 1 and 24)
        -af N: resample animations at N Hz (default: 30)
        -ac: keep constant animation tracks even if they don't modify the node transform

Scene:
        -kn: keep named nodes and meshes attached to named nodes so that named nodes can be transformed externally
        -km: keep named materials and disable named material merging
        -ke: keep extras data
        -mm: merge instances of the same mesh together when possible
        -mi: use EXT_mesh_gpu_instancing when serializing multiple mesh instances

Miscellaneous:
        -cf: produce compressed gltf/glb files with fallback for loaders that don't support compression
        -noq: disable quantization; produces much larger glTF files with no extensions
        -v: verbose output (print version when used without other options)
        -h: display this help and exit

We'll need a mapping between gltf-pipeline arguments and gltfpack arguments. To start I would suggest

--gltfpack.optimize

true by default if any other gltfpack options are set. If --gltfpack.quantize and --gltfpack.compress are not passed in, -noq should be passed in so that the model doesn't get quantized.

--gltfpack.quantize

optimize and apply quantization. No gltfpack options required. The glTF will have the KHR_mesh_quantization extension

--gltfpack.compress

compress the glTF. Pass in either the -c option or -cc option (the -cc option should be fine, are there any drawbacks?) The glTF will have the EXT_meshopt_compression extension. Mutually exclusive with --gltfpack.quantize.

--gltfpack.combine (might need a better name)

combines similar nodes, meshes, and materials. Pass in -mm. When this option is not used pass in -kn, -km, and -ke so node hierarchies and extras are preserved.

Because gltfpack reads and writes from glTFs on disk gltf-pipeline will need to save out the gltf before calling gltfpack. gltfpack can read and write to the same file on disk so no temporary files should be required.

gltfpack also includes KTX2+Basis texture compression, so we can add another CLI option to gltf-pipeline

--basis

pass in -tc to gltfpack. I'm not sure if it's possible to apply texture compression and not also optimize the model.

Other:

• Draco and gltfpack options should be mutually exclusive
• gltfpack should exit with an error message if the glTF already has KHR_mesh_quantization, EXT_meshopt_compression, or KHR_draco_mesh_compression
• I started a skeleton branch here: https://github.com/CesiumGS/gltf-pipeline/tree/gltfpack
• Need to coordinate gltf-pipeline's own --separate and --separate-textures with gltfpack

CC @YoussefV @zeux (hopefully I've represented everything accurately here)

[zeux]

-cc option should be fine to use as a default instead of c, gltfpack may drop the distinction long term.

-c/-cc is complementary to gltfpack.quantize, it's recommended to use both (-c without quantization currently doesn't produce high compression ratios; this might change in the future, although it's unlikely to), because of this gltfpack defaults to quantization. I'd recommend defaulting quantize to true, this is similar to Draco compression (Draco always quantizes, although it doesn't use KHR_ extension for that).

There's some discussion in https://github.com/zeux/meshoptimizer/issues/164 on a possible JS API. It should be possible to implement a library-like interface to gltfpack that uses Emscripten's MEMFS to work with files (so API would still be path-based but the paths would be read/written using MEMFS) - is that of interest here?

[lilleyse]

@zeux maybe an enum works better here:

• Optimizations only, lossless (-noq)
• Optimizations + quantization (no options)
• Optimizations + quantization + compression (-cc)

There's some discussion in zeux/meshoptimizer#164 on a possible JS API. It should be possible to implement a library-like interface to gltfpack that uses Emscripten's MEMFS to work with files (so API would still be path-based but the paths would be read/written using MEMFS) - is that of interest here?

Yes, definitely of interest.

[donmccurdy]

I'd be interested in picking up EXT_meshopt_compression in glTF-Transform as well, and a JS API would certainly help with that. Or WASM is fine too, as long as it runs both in the browser and in Node.js. The tiny meshopt decoder is great, not sure if something similar is feasible for the encoder.

I think my one complaint with the current gltfpack API (for this purpose) is that it does everything by default. For an integration in something like glTF-Pipeline you may want to be able to easily isolate a single change: just quantize, just consolidate nodes, or just convert textures to KTX. For a JS API I think you'd want these things to be opt-in rather than opt-out.

[donmccurdy]

Following up here – @zeux has published a meshoptimizer JS API and NPM package. I've integrated that (both encode and decode) with glTF-Transform, in case the code is helpful as a reference:

• packages/extensions/src/ext-meshopt-compression

Notably, quantization should be applied to the geometry before compressing with Meshopt (example).