Demos don't work on ios

[gkjohnson]

[gkjohnson]

cc @setpixel - are the demos still not working on Safari on iOS? I'm seeing that they work on my iPad Pro Safari / Chrome right now. I don't recall if it was broken previously.

Thanks!

[VrTech]

cc @setpixel - are the demos still not working on Safari on iOS? I'm seeing that they work on my iPad Pro Safari / Chrome right now. I don't recall if it was broken previously.

Thanks!

Works well on iPad but not iPhone apparently. I tested both

[gkjohnson]

@VrTech Great, thanks! Are there any console error logs on the iphone at all?

[thomasaull]

I tested in on my iPhone and checked the logs with the web inspector extension but there were no logs (error or info) unfortunately.

[thomasaull]

Oh okay, so I tested the first demo (Basic setup example) which didn‘t work (black screen while rendering samples) but another demo (Material test orb) is working on my iPhone

[gkjohnson]

Thanks for checking! Interesting that it worked on at least one of the examples...

but there were no logs (error or info) unfortunately.

This makes it hard to track down. Hopefully someone from Webkit can chime in from the tweet and we can get more insight. I suspect that something needs to be fixed in webkit.

[thomasaull]

@gkjohnson I tried almost every current example and here's the result what works on my iPhone and what shows a black screen for the path traced view. All testing was done on an iPhone Xs with iOS 16.1

Scene	Works?	Notes
Basic Setup Example	❌
Physically Based Materials	✔
Lego Models	✔
Interior Scene	❌	shows only a black screen for the realtime view also
Depth of Field	❌
Skinned Geometry Support	❌
Morph Target Support	❌
Area Light Support	❌	shows only a black screen with a gizmo for the realtime view
Spot Light Support	❌
Material Test Orb	✔
Transmission Preset Orb	❌
Model Viewer Fidelity Scene Comparisons	❌	Safari completely crashes after the Building BVH step an reloads the entire page
Physical Material Database	❌	Does not load correctly, there is an actual error message in the console though: Reference Error: Can't find variable: hideInfo

Not sure if it helps or you/someone can find a pattern there…

Also I'm not sure if the Web Inspector Extension on iOS shows logs reliably. I did check in Browserstack too but there aren't any logs showing up either.

[gkjohnson]

Thanks @thomasaull! I didn't realize that only some of the examples weren't working. It's odd that no errors are logged... I'm surprised that the "Basic Setup" example isn't working. It would be good to know if there's another version of the project that does work. Then we can bisect to find where it broke. It's possible 0.0.7 was working if you want to give that a try.

Does not load correctly, there is an actual error message in the console though: Reference Error: Can't find variable: hideInfo

This was an error in the JS - it should be fixed now.

Safari completely crashes after the Building BVH step an reloads the entire page

The default dragon scene for the model viewer is pretty intensive so that could be one reason it breaks.

[thomasaull]

Hey @gkjohnson I tested the „Basic Example” from v0.0.7 to v0.0.3 but path tracing was not working on my phone for any of these versions. No idea 🤷‍♂️

[gkjohnson]

@thomasaull can you check if the following demos work?

• Physical Material Database now that I've fixed the JS error
• Simpler Avocado scene in Viewer Test

[thomasaull]

@gkjohnson First demo does not work (black screen while path tracing). Second demo works but occasionally crashes safari (maybe 1 in 4 times). Attached is da quick screencast:

RPReplay_Final1674393423 (1).webm

[robertoranon]

this is the situation I currently see with the latest iOS (17.0.3, on iPhone 13 mini) and iPadOS (17.0.3, on iPad Pro M1):

• any example using FEATURE_MIS > 0 crashes
• once MIS is disabled, examples that use an environment map for lighting run without errors, but show a black canvas (they work instead on safari/chrome macOS ventura), as if there was no light

Interestingly, on iOS / iPadOS the index.js example shows a black screen both when the path-tracer is running (still camera) and when three.js is rendering (when moving the camera).

Normally, in threejs you would read the RGBE envmap creating a half-float texture, and then the pmrem code processes it to a half-float render target (which is then the texture used as scene.environment value to associate the env map to each material). Instead, three-gpu-pathtracer, after the pmrem step, renders (with blur) the result to a float32 buffer, then reads back to a float32 data texture, which is then assigned to scene.environment.

In the past, I've seen situations where moving from float to half-float textures with iOS would fix issues, but I cannot find right now any reference backing up that.

[gkjohnson]

In the past, I've seen situations where moving from float to half-float textures with iOS would fix issues, but I cannot find right now any reference backing up that.

Can you try removing the "setType( THREE.FloatType )" call here to see if that fixes at least the three.js rasterized rendering? Looks like the use of FloatType for loaded data was added in #404.

[robertoranon]

It does not fix the three.js rendering, but if you also change the animate function as follows, then it does:

function animate() {

	requestAnimationFrame( animate );

	camera.updateMatrixWorld();
	pathTracer.update();

	if (pathTracer.samples < 1) {

		renderer.render(scene, camera);

	} else {

		renderer.autoClear = false;
		blitQuad.material.map = pathTracer.target.texture;
		blitQuad.render(renderer);
		renderer.autoClear = true;

		samplesEl.innerText = `Samples: ${Math.floor(pathTracer.samples)}`;
		
	}

}

Took me a while to figure that out (I had to basically deconstruct and reconstruct the example), and I don't understand why it fixes three.js rendering (or why the previous version worked on desktop)

[gkjohnson]

To be clear you have to change both? Or is changing the animate function enough meaning that the setType( THREE.FloatType ) call doesn't matter.

I don't understand why it fixes three.js rendering (or why the previous version worked on desktop)

I'm not sure what you mean here. The rendered pathtracing buffer is normally composited on top of the three.js rasterized scene so there's coherence while the pathtraced "tiles" fill in.

[robertoranon]

both. If you leave the float type, you get something rendered, but totally wrong illumination

[gkjohnson]

Gotcha - thanks! Can you post a screenshot of the three.js rendering with and without the setType( FloatType ) call? And then it's probably worth removing that, then, if it's going to cause issues.

The EquirectHdrInfoUniform also relies on Float32Array which I imagine is causing issues, too. The next steps will probably be re-adding lines to the path tracing shader to see where it breaks on the iphone. The environment map may just be part of the issue, though.

[robertoranon]

Sure, here they are. With setType( FloatType )

without:

Also, I managed to make the environment map lighting work on iOS also in the path tracer rendering, apart from MIS, by making changes to EquirectHdrInfoUniform.js and BlurredEnvMapGenerator.js to make everything using HalfFloat textures, here's a screenshot, and I can make a PR, if you're interested

[gkjohnson]

@robertoranon Awesome! Thank you. PRs for both fixes would be great.

[robertoranon]

made a single PR for both fixes here: https://github.com/gkjohnson/three-gpu-pathtracer/pull/466