CubeTexture orientation

[fabienrohrer]

In my app, I would like to rotate the CubeTexture that defines the scene background and the environment map reflections by 180° along the vertical y-axis. My rotation is defined by the VRML and X3D specifications about the background fields.

I tried to modify several parameters like the mapping, flipY and rotation Texture fields. However I didn't found any good way to do so.

• Is there some obvious way to do so that I missed?
• Any chance to see this coming in three.js?
• If I would like to implement it, what parameters would you like to see (for example Texture.flipX and CubeTexture.flipZ)? Would you be interested in this feature?

[mrdoob]

You mean being able to rotate scene.background? If so... yes, definitely interested.

We probably need a new API though...

scene.background = new THREE.Background( cubeTexture );`
scene.background.rotation.y = Math.PI / 2;

[looeee]

I'm struggling with exactly this right now. Trying to get baked ground shadows to match up to the sun direction from a cubemap. It seems like somewhere along the way my cubemap is being flipped 180 degrees. It would be great if I could just fix that here rather than re-rendering my cubemap multiple times and trying to figure out where that's happening.

[WestLangley]

@fabienrohrer Try

scene.rotation.y = Math.PI;

If that is not working, please provide a live example to demonstrate exactly what you are doing.

[mrdoob]

Oh, that works? 😮 The camera must be a child of the scene though?

[WestLangley]

The camera must be a child of the scene though?

I don't think that matters.

[fabienrohrer]

@mrdoob this is exactly what I need too :-)

@WestLangley:

Rotating the overall scene is overkilled and may cause many side effects in my app.

I would like to be able to rotate the background only, and for sure, the environment maps should work the same way.

A live demo is not possible to do, because the API is missing. It's a matter to take any example having a background cubemap (like https://threejs.org/examples/#webgl_materials_cubemap) and be able to rotate only the background around the y-axis. Or more generally, to apply any rotation to the background.

Being able to apply a custom rotation matrix / quaternion to any cubemap seems to be the most generic solution for me.

[WestLangley]

three.js is careful to ensure that material reflections are consistent with the material envMap, which is defined in the world space coordinate system. Allowing users to rotate the scene background would result in a background that is inconsistent with material reflections.

Why don't you create the correct world-space cube map in your app?

[mrdoob]

The camera must be a child of the scene though?

I don't think that matters.

I tried doing scene.rotation.y ++ in the console in webgl_materials_standard and the gun rotates. Then I tried doing scene.add( camera ) and then if I do scene.rotation.y ++ it does what we're after, but then the camera controls get messes up 😁

If we introduce a THREE.Background object, we could add API to handle this, plus in the renderer we can also introduce the idea of:

var envMap = material.envMap;
if ( scene.background && scene.background.isBackground && envMap === null ) {
    envMap = scene.background.texture;
}

[WestLangley]

I said

Try scene.rotation.y = Math.PI; If that is not working,...

That means it may not work. :-)

It depends on the use-case. I tried with OrbitControls and it worked fine.

unrelated: the webgl_materials_standard example should be using orbit controls, anyway IMO.

[looeee]

You mean being able to rotate scene.background? If so... yes, definitely interested.

It makes more sense to do this at the texture level, otherwise any IBL lighting will not match the background, I assume.

We allow rotation of normal textures, is there any reason why we wouldn't want to do the same for cube textures? Is it too complex to implement?

Why don't you create the correct world-space cube map in your app?

This is very time consuming. I've been working on generating cube maps over the last couple of days, and making changes then re-rendering the cube map takes me about 15-20 minutes for each change.

Another potential interesting use case for this is that you could change the direction of environmental lighting in real time. I'm currently combining a PMREM environment map with a directional light for real-time shadows, and using the map as a skybox, which gives a fairly decent daylight effect. Obviously there's a limit to what you could do without changing the color of the environment map, but you could simulate at least a couple of hours of daylight passing by rotating the map in sync with the directional light while adjusting the .envMapIntensity.

[fabienrohrer]

My concern is also about performance. Currently I implemented a hack about to rotate the top and bottom texture and swap the other ones. It takes about 10ms to rotate a 1024x1024 texture in JS.

[omichel]

Alternatively, could it be possible to have only two options for the CubeTexture orientation, the one you currently have and another one (with the 180 degree rotation) that would be friendly with the other standard largely used in 3D formats (X3D, VRML, etc.). We could probably easily adapt the consistency with material reflections to handle both formats?

[makc]

This sounds familiar #11103

[fabienrohrer]

If I would implement CubeMap.rotation = matrix3, would you merge in r105?

[WestLangley]

My concern is also about performance. Currently I implemented a hack about to rotate the top and bottom texture and swap the other ones. It takes about 10ms to rotate a 1024x1024 texture in JS.

And how often are you doing this in your app?

[fabienrohrer]

@WestLangley I just give you a detailed answer here: https://github.com/mrdoob/three.js/pull/16507#discussion_r286337864

And how often are you doing this in your app?

Not much. Once at loading per client, and once each time the background is changed (very rare case). A DEFINE would also do the job.

[WestLangley]

My concern is also about performance ... It takes about 10ms.

And how often are you doing this in your app?

Not much. Once at loading per client, and once each time the background is changed (very rare case).

Sorry, I am not following the logic of that.

[fabienrohrer]

Unfortunately, @WestLangley doesn't want to merge my modification about the CubeTexture.rotation because of the runtime overhead (a supplementary mat3 multiplication). In this case, I'm running out of ideas to solve this cleanly in threejs. If threejs contributors won't solve this, please close this issue.

[FishOrBear]

I wait for the implementation of this feature.

[fabienrohrer]

@FishOrBear Unfortunately, the threejs maintainers decided to not implement this. You could apply this unmerged patch in a fork, if you would like to have this in your project:

https://github.com/mrdoob/three.js/pull/16507

[Mugen87]

@mrdoob Maybe we can reconsider this feature requests in #18157?

[FishOrBear]

I am using threejs to make an application similar to AutoCAD, so we use xy to sit as a plane.

The current CubeTexture conflicts a bit with ours.

[Mugen87]

BTW: With Babylon.js, it is at least possible to rotate a skybox around the y-axis:

https://www.babylonjs-playground.com/#UU7RQ#447

The engine supports this by transforming the reflection vector in the fragment shader. So exactly the approach which was not accepted in #16507. Code from Babylon.js fragment shader:

https://github.com/BabylonJS/Babylon.js/blob/1c4627141f83c08fe4a7e30e2621c916b4e6c9c9/src/Shaders/ShadersInclude/reflectionFunction.fx#L114-L117

[Mugen87]

@mrdoob I think we should implement this feature. There is another request in the forum:

https://discourse.threejs.org/t/rotate-a-scenes-background-skybox-texture/12199

I doubt that a single additional matrix/vector multiplication per fragment will noticeably affect the performance.

[fabienrohrer]

I still have the same issue in my app, and the hack to solve it is ugly and so costly... I also recommend to accept this improvement.

[looeee]

I also would love to see this feature added.

It seems like the blocker is potential decrease in performance, but that should be easy to test. We can try adding it, and if there's an unnaceptable performance drop we can remove it again.

[DavidPeicho]

Same here, could use this feature instead of re-implementing my background rendering :)

I am not sure to follow on the performance issue. Nothing prevents us to generate the ray direction directly in the vertex shader? Then it would cost as much as any other modelMatrix transform.

[capnmidnight]

In my case, I could use this feature because I'm rendering cubemaps captured by cameras and uploaded by users. The photos aren't always oriented to north correctly, so they need a manual correction after upload.

One way that I've done this is to store the rotation as a variable, apply it to the scene just before rendering, and then resetting the scenes' rotation to the identity quat just after rendering.

This causes two unnecessary updates to all my scene objects' world matrices. It also means I can't do any position updates in onBeforeRender. But it keeps the scene in a normal state otherwise.

[wmcmurray]

Btw (since no one mentioned it), it seems to be possible to make a "classic" skybox that rotates using an array of 6 materials.

something like this :

const urls = [
  '/assets/skybox/right.png',
  '/assets/skybox/left.png',
  '/assets/skybox/top.png',
  '/assets/skybox/bottom.png',
  '/assets/skybox/front.png',
  '/assets/skybox/back.png',
];

const materials = urls.map((url) => {
  const texture = new THREE.TextureLoader().load(url);

  return new THREE.MeshBasicMaterial({
    map: texture,
    side: THREE.BackSide,
    fog: false,
    depthWrite: false,
  });
});

const skybox = new THREE.Mesh( new THREE.BoxBufferGeometry(10000, 10000, 10000), materials );
scene.add(skybox);

skybox.rotation.y = Math.PI / 2;

As demonstrated here : https://woodenraft.games/demos/skybox-rotation-threejs.html

[capnmidnight]

@wmcmurray You'll have to move this skybox with the camera to make it work right, and you'll have to make sure that your box dimension X < Math.sqrt(0.75 * camera.far * camera.far) so the corners always fit in the view, but that seems easier than my hack of rotating/unrotating the scene.

[DavidPeicho]

Should we re-open a PR for that? There are two use cases and if we don't want to affect performance we can use a shader define:

• Use case 1: environment is pre-computed once and shouldn't need to be rotated in the shader. This could be done by the user either on the CPU, or even on the GPU if he wants to
• Use case 2: when working on a viewer, it's quite comon to rotate the background in real time using the mouse for instance.

In order to prevent the user with use case 1 to be affected by the matrix multiplication, maybe we can simply add a flag like:

renderer.dynamicBackground = true;

and compile the background shader accordingly?

In the meantime

As @wmcmurray and @capnmidnight pointed, you can basically re-code your own background rendering and it's not that difficult. Just be extra careful with the viewing plane (near and far values).

Instead of using MeshBasicMaterial, you can also directly create your own shader extending ShaderMaterial and sample the cubemap.

[wmcmurray]

Just to add on top of my previous comment, the method proposed will generate 6 draw calls per frame (1 per material) this is not very good... along with specificities mentioned here. 😕

I ended up solving all those issues 😏 with this approach :

1. I took all my skyboxes composed of arrays of materials, (I actually have many, one for the sky, one for stars, one for clouds, etc) and I put all of them in a different scene.
2. I then create a WebGLCubeRenderTarget along with a CubeCamera that renders into this target. Then I add this camera into the scene with my skyboxes.
3. I then set the background of my original scene to the texture property of this render target.

From there, I can just call CubeCamera.update() to generate a cube texture of the whole scene with skyboxes, this texture will be used as the background of my game scene, it will be rotated properly and it will be rendered with only 1 draw call per frame ! 👍🏼 You don't even have to move the skybox with the camera since the CubeCamera is fixed in the skybox scene.

You can also pass that texture to a PMREMGenerator to generate a scene.environment texture, this will give you dynamic envMap reflections on all PBR materials, for "free" ! 🤯

This method works very great when you just want to generate your background once, but it also works with a dynamic background that moves over time ! In my case, the sky rotates slowly so I can get away with updating the skybox scene texture about 8 times per seconds and it still seems fluid enough 🙂 It's also super easy to tweak the texture resolution to reduce the work load.

[sdraper69]

I have noticed that sketchfab and playcanvas have this function available, are we (and by that I mean someone cleverer than me) not able to see how they do it?