[iOS] The page is refreshed when uploading certain objects from Sketchfab

[Softvision-RemusDranca]

[Affected Versions]:

• Safari 15
• Chrome 93

[Affected Platforms]:

• Iphone SE2

[Prerequisites]:

• A room is created and opened on https://hubs.mozilla.com/.

[Steps to reproduce]:

1. Tap on the "Place" button.
2. Click the "3D Model" button.
3. Select for example the "St Olaf" object.
4. Observe the behavior.

[Expected results]:

• The object is spawned into the room.

[Actual results]:

• The Hubs page is refreshed.

[Notes]:

• Attached is a screen recording of the issue: link.

┆Issue is synchronized with this Jira Task

[takahirox]

Thanks for the report.

Unfortunately I couldn't reproduce the problem on my iPhoneX.

The model has 8192x8192 texture image. I guess it's too huge for iPhone SE2?

I first want to check WebGL stats on iPhone SE2. Would you mind if accessing https://webglreport.com/?v=2 on your iPhone SE2 and report "Max texture size" to me?

And I made a simple test which just renders the model with Three.js. Can you access the following URL and check if you see the same problem?

https://takahirox.github.io/ios-render-test/

If you face the same problem, the problem may not be Hubs specific issue but may be a device side problem.

[Softvision-RemusDranca]

@takahirox The Max Texture Size on my Iphone SE2 is "16384". I've encountered the same issue with the test page after refreshing the window (I did not encountered it when first accessing the page). At the first reload I reproduced the same issue and after some other page refreshes I got the message "A problem repeatedly occurred..". Here is a screenshot :

[Softvision-RemusDranca]

@takahirox I've encountered the same behaviour while changing scenes. Here is a video. I reproduced it while having multiple tabs opened and changing the scene. Not sure, but could this be a safe mechanism on iOS, where when a tab consumes to many resources it will reload it? If it's the case it could be related to the high CPU usage issue...

[takahirox]

@Softvision-RemusDranca

Thanks for the test. iPhone SE2 WebGL stats looks covering 8192x8192 texture image.

The fact that you can reproduce the problem on the simple test means the problem is not Hubs specific but maybe is from the device limitation. I have an impression that it crashes when uploading huge textures.

If my speculation is true, potential solutions may be

1. Prohibit huge textures. If I'm right we do for too may polygons models. We may be able to prohibit huge textures. It would be good if configurable in the scene settings?
2. Dynamically resizing (huge) textures. And we may be able to use resized texture as lower LOD if we introduce LOD feature. Related: #3565 #4614
3. There might be a chance that using compressed textures resolves the problem because of less memory consumption? We may need to dynamically compress for assets which don't use compressed textures tho.

[takahirox]

I've encountered the same behaviour while changing scenes.

Unfortunately I couldn't reproduce on my iPhoneX. I watched the video and have an impression that it crashes when first rendering the new scene (meaning when uploading textures and vertices).

I reproduced it while having multiple tabs opened

How many tabs did you open?

If it's the case it could be related to the high CPU usage issue...

It can be. If high CPU usage and huge textures are related, I guess decoding huge texture image crashes browsers?

Can the problem be reproducible with any scene? And any console warnings/errors when browsers crash?

[Softvision-RemusDranca]

@takahirox I had 2 tabs additionally opened, Youtube and Wikipedia. In today's attempts I was only able to reproduce the issue with the "Hubs School v1.0" scene. Unfortunately I don't have a cable so I can connect the Iphone to other devices to check for logs at the moment. Could there be any app for this or other methods than connecting to a macOS device?

[takahirox]

@Softvision-RemusDranca

I had 2 tabs additionally opened, Youtube and Wikipedia.

Hm, that usage sounds like very normal usage.

In today's attempts I was only able to reproduce the issue with the "Hubs School v1.0" scene.

Thanks, I will ask the artist team about the model.

Could there be any app for this or other methods than connecting to a macOS device?

Unfortunately no idea. And I guess it may be impossible due to security stuffs.

So, my current impression is still https://github.com/mozilla/hubs/issues/4669#issuecomment-929537528. It might be ok to suggest to use low-poly and low-resolution textures for now. And the problem will be resolved sooner or later when newer models will be released and iPhone users will move to them, or when we will do more advanced optimization like dynamic texture optimization.

[takahirox]

The artist team mentioned "Hubs School v1.0" uses several 2048x2048 textures. I guess they can cause the crash. And they seem to optimize a bit when releasing v2.

[takahirox]

Sounds like the same issue as #3332?

[brianpeiris]

I was able to repro this on my iPhone 11, using the "St Olaf" Sketchfab model linked in the description. It repro'd in three different ways, once it refreshed the page, another time it stalled Hubs, and another time it rendered as a black model without textures.

It's certainly a memory issue due to the large texture on that model. The solutions mentioned above seem like good ideas. We might also want to see if Sketchfab can give us lower-quality versions of files in the first place.

[takahirox]

Confirmed that the problem is still reproducible on my iPhoneX iOS 15.4.1 Safari. #5295 may be a solution.

[takahirox]

I want to raise the priority a bit because I faced this problem when I tried to enter our standup room from my iPhoneX Safari. And also I was kicked out from a room when someone placed a huge texture object.

[takahirox]

I made an example which resizes the texture image from 8192x8192 to 512x512 on the fly.

https://takahirox.github.io/ios-render-test/texture_resize.html

The page is no longer refreshed or crashed but renders without any troubles. I worried if it can be crashed when resizing the texture with canvas but it didn't happen. It seems that the problem may be WebGL specific.

I feel positive about dynamic texture resize.

[brianpeiris]

Nice one! That code is much simpler than I thought it would be! It would be interesting to see if this causes a significant hitch if there are many models in the room, since the resize happens on the main thread. It may not be a problem for models that are in the scene or pinned in the room, since we could do that when the room loads, but it may be a problem when you add objects to the room after entering.

[takahirox]

I didn't test well enough yet but my current impression is the main thread blocking time of image resize and smaller texture upload is shorter than the one of huge texture upload.

Even if the hitch caused by the resize can be a problem, we may be able to think of offloading it from the main thread by using OffscreenCanvas (and createImageBitmap?) tho I'm not sure if it's doable on all major platforms.

Another concern is resizing compressed texture might be impossible? Compressed textures might be less problematic even if they are huge tho.

[brianpeiris]

Yeah, doing it off main thread may be an additional enhancement. Glad to hear that it may not cause a big hitch. This is certainly better than crashing anyway!

[takahirox]

I want to discuss how to expose the configuration in #5295

[takahirox]

The problem seems to happen with huge textures or too many textures. For example the problem happened with 202 textures whose sizes are up to 1024x1024 in a room. (Perhaps it can be caused based on total (VRAM) memory usage.)

For huge textures, #5437 can be workaround. But it might not be a solution for too many textures. We may need to think of limiting the number of textures, disabling textures, or optimizing on assets creation end.

[takahirox]

Reading this

https://bugs.webkit.org/show_bug.cgi?id=232357

There might be a chance that ImageBitmap is involved in the problem. I would like to test disabling ImageBitmap.

[takahirox]

Unfortunately the problem is reproducible even though I disabled ImageBitmap on iOS Safari.

[takahirox]

I made a simple test reproducing the problem.

https://takahirox.github.io/ios-render-test/simple.html

The test just upload 8192x8192 texture(s) with gl.texImage2D(). On my iPhoneX Safari (iOS 15.5), the Safari can crash or the page can be unexpectedly refreshed by uploading 3 or 4 textures.

Can you who have iOS devices test on your devices, too?

I will share this simple test with the webkit team.

[brianpeiris]

Can confirm, I can get Safari to crash or refresh with that test on my iPhone 11 with iOS 15.4 and Safari 15.4. I'll install updates and report back.

Interestingly, if I use Chrome on my Samsung S21, I can't get it to crash, but Firefox on the same phone crashes with 6 or 7 textures.

[j-conrad]

Anything beyond 2 textures refreshes the page on my iPhone11 using Firefox.

Anything beyond 1 texture refreshes the page on my iPhone11 using Safari or Chrome.

iPhone11, iOS 15.5

[brianpeiris]

Not surprisingly, as Jim has shown, my iPhone 11 also still crashes after updating to iOS 15.5 (and Safari 15.5)

[wsxiaoys]

iOS 15.4.1, 12 Pro Max, Safari

1. imagebitmap on / mipmap on: crash with 3 or more.
2. image bitmap off / mipmap on: crash with 8 or more.
3. image bitmap off / mipmap off: crash with 7 or more.

[vincentfretin]

No crash with 10 textures on Safari iPadOS 15.5 iPad 2018 6th gen.

[vincentfretin]

Just to be sure, what should be a successful output? I only have

Load image #1 is started
Load image #2 is started
...
Load image #10 is started

at the end. Is this successful?

[vincentfretin]

From the script apparently not, it should end with "Test finished". Oh it takes a really long time, 12 sec to upload only one texture. I'll report back when I tested several textures and waited long enough, sorry.

[vincentfretin]

Ok, here it is for Safari iPadOS 15.5 iPad 2018 6th gen (WEBGL2 MAX_TEXTURE_SIZE: 16384), it supports 0-3 textures (it worked with 1, 2, 3 the first time I tested, the other attempts crashed or waiting indefinitely) with ImageBitmap, and 4 textures without ImageBitmap. The iPad restarted several times during my tests.

• with Use ImageBitmap, with Generate mipmaps, Texture num 3, the test is successful (but on other attempts the page reloaded, even with 2 or 1)
• with Use ImageBitmap, with Generate mipmaps, Texture num 4, the page reload
• Use ImageBitmap unchecked, with Generate mipmaps, Texture num 4, test is successful
• but when I refreshed the page, then the iPad restarted....
• Use ImageBitmap unchecked, with Generate mipmaps, Texture num 5, the page reload
• Use ImageBitmap unchecked, Generate mipmaps unchecked, Texture num 5, the page reload

[takahirox]

Thanks all for the tests.

I filed an issue at webkit bugzilla https://bugs.webkit.org/show_bug.cgi?id=241478