Idea: Fidelity Tests like model-viewer has

[hybridherbst]

This is an awesome resource.

I'm not sure if you're aware of that already, but model-viewer has a fidelity test result page and a format for how to integrate other renderers / engines: https://modelviewer.dev/fidelity/

They have a pretty simple JSON format for how IBL, camera etc. are specified to ensure matching conditions between renderers.

For automating along with what model-viewer already does, renderers would need to be added here: https://github.com/google/model-viewer/tree/master/packages/render-fidelity-tools/src/components/renderers

[cx20]

Sorry for the delay in replying. Thanks for the great ideas. I too wish I could do a validation like model-viewer. However, I lack the skills and time to do so.

BTW, I am making the following efforts to improve testing with gltf-test. https://github.com/cx20/gltf-test/issues/68

[hybridherbst]

Thanks for responding! I think the interesting thing would be that the validation / image generation could run as part of what model-viewer already does. Maybe @elalish has some insights as well into whether wider comparisons would be desired inside model-viewer.

Regarding #68, that's a good start and something definitely lacking from glTF currently (defining which IBL, exposure, tonemapping a renderer should use)! Seems like model-viewer actually had similar issues, here's what looks like a shell script to prepare an IBL for filament rendering: https://github.com/google/model-viewer/blob/master/packages/render-fidelity-tools/test/renderers/filament/prepare-ibl.sh

[elalish]

We'd be thrilled to have more renderers added to our fidelity test suite. It's not generally a heavy lift, especially if your package is on npm. If you're interested I'd be happy to share some pointers (like old PRs that added other renderers).

[cx20]

@ybridherbst Sorry for my late reply.

defining which IBL, exposure, tonemapping a renderer should use

Thank you for sharing your insights. I have had a lot of advice from the Filament and Khronos peoples that if the exposures and tonemapping are not set up correctly, the results displayed will be different. I need to understand more about exposure and tonemapping besides IBL. https://github.com/google/filament/issues/3366#issuecomment-747752013 https://github.com/KhronosGroup/glTF-Sample-Models/pull/340#issuecomment-1065797662

@elalish I would like to understand how the fidelity test works. I would be glad to know if you have any other renderer samples or pointers that would be helpful. BTW, My skill set is old and I may need to start by getting familiar with TypeScript.

[cx20]

Thanks again to everyone who has been advising me on this repository. @hybridherbst @elalish @emackey @donmccurdy @willeastcott @slimbuck @emadurandal @Reon90 @06wj @sonygod @DrX3D

I'm aware that this repository mechanism is a legacy mechanism. I don't want to break links such as URL references from outside, so I'm thinking of preparing a new repository. I have set up a discussion room. If you have any other ideas, I would appreciate your contribution.

https://github.com/cx20/gltf-test/discussions

Thanks!

[Reon90]

@cx20 What is the status of current repo? Are you planning to continue contributing or move your effort to new one?

[cx20]

@Reon90 I'm planning a new (modern) repository, but I think it will take some time. Until then, I think we will continue to update this repository.

See below for our plans. https://github.com/cx20/gltf-test/discussions/560#discussioncomment-2577491