Wrong background color using Open3DScene

[frebro83]

Checklist

• [x] I have searched for similar issues.
• [x] For Python issues, I have tested with the latest development wheel.
• [x] I have checked the release documentation and the latest documentation (for master branch).

Describe the issue Using Open3DScene set_background seems to result in slightly wrong color. Attempting to match the 1E1E1E color of visual studio code results in 0D0D0D, am I missing something?

Steps to reproduce the bug

import open3d as o3d

class ColorTest:
    def __init__(self):
        self.window = o3d.visualization.gui.Application.instance.create_window(
            "ColorTest", 640, 480)
        self.scene = o3d.visualization.gui.SceneWidget()
        self.scene.scene = o3d.visualization.rendering.Open3DScene(
            self.window.renderer)
        
        # Following line is supposed to set background color to 1E1E1E,
        # alpha is ignored.
        self.scene.scene.set_background([30/255, 30/255, 30/255, 1])
        
        # Show something on screen
        self.scene.scene.show_axes(True)

        # Setup camera towards origin
        bbox = o3d.geometry.AxisAlignedBoundingBox([-1, -1, -1],
                                                   [1, 1, 1])
        self.scene.setup_camera(60, bbox, [0, 0, 0])

        self.window.add_child(self.scene)

def main():
    o3d.visualization.gui.Application.instance.initialize()
    ColorTest()
    o3d.visualization.gui.Application.instance.run()

if __name__ == "__main__":
    main()

Error message None

Expected behavior FEngine (64 bits) created at 0000022BA0741050 (threading is enabled) FEngine resolved backend: OpenGL

Window appears with a background darker then expected (0D0D0D) with the axes shown.

Open3D, Python and System information

- Operating system: Windows 10
- Python version: Python 3.8
- Open3D version: 0.17.0
- System architecture: x86_64
- Is this a remote workstation?: no
- How did you install Open3D?: pip

Additional information No response

[theNded]

@errissa this gamma correction has been disturbing me for a while too... I usually have to manually photoshop the background before putting a screenshot in my paper. Is it possible to find a way to both use PBR and retain a white background?

[errissa]

@frebro83 You're not doing anything wrong. By default the rendering pipeline does a number of post-processing steps which include color grading and tone mapping. So, the background color will almost never match exactly what you set it to be. The easy way to fix this if you need exact colors is to disable post processing in the _init_ method: self.scene.scene.view.set_post_processing(False). set_post_processing can be called anytime - you can keep it on when you don't care, but then disable it when you need perfectly accurate background color.

To make the effect more obvious I changed the background to white and captured these images.

Default rendering (with post processing):

Post processing disabled:

@theNded The method described above should work for you but may or may not be easy to incorporate depending on how you're using the visualizer. Feel free to reach out on Discord or we can talk about it on Friday.

[frebro83]

So, I tested to set_post_processing as suggested and it worked for light backgrounds but I did not get any visual difference for dark backgrounds. In general I'd like to be able to turn off the gamma/color correction on background and/or skybox while keeping it on for any objects.

It is of course a bit backward and extra work, but I can always compensate for the gamma correction in the dark spectrum and in case I need to go close to white I could attempt to use the set_post_processing, depending on how much it will interrupt the object shading.

Thank you for the answer!

[errissa]

The color grading adapts to scene content so it is, generally speaking, less obvious for darker scenes. Unfortunately, the post processing effects are screen space effects so it's impossible to disable them for just the background.

[frebro83]

As the post_processing wasnt enough to disable this color grading for darker scenes, are there more settings that can be used to control this?

[errissa]

You can control the color grading explicitly. Instead of disabling the post processing you can do the following:

import open3d.visualization.rendering as rendering

...

cg_settings = rendering.ColorGrading(rendering.ColorGrading.Quality.ULTRA, rendering.ColorGrading.ToneMapping.LINEAR)
self.scene.scene.view.set_color_grading(cg_settings)

The first constructor parameter for ColorGrading takes a quality (LOW, MEDIUM, HIGH, ULTRA) and the second a tone mapping algorithm (LINEAR, ACES_LEGACY, ACES, FILMIC, UCHIMURA, REINHARD). The default is Uchimura. You would want LINEAR since it avoids non-linear transformations of the color. The ColorGrading class also has a temperature property which sets the white balance color temperature. There are many more options for configuring the color grading but it appears that other than the ones I just noted they are not exposed in the Python API.

[frebro83]

Ok, great. Thank you!

[yanghtr]

@errissa , @frebro83 I find that the results are still not very perfect with the proposed solutions. Below is an example for illustration.

import open3d as o3d
import open3d.visualization.rendering as rendering
from matplotlib import pyplot as plt

render = rendering.OffscreenRenderer(720, 720)

# setup geometry and camera
sphere = o3d.geometry.TriangleMesh.create_sphere(1.0)
sphere.compute_vertex_normals()
mat = rendering.MaterialRecord()
mat.base_color = [0.7, 0.7, 0.7, 1.0] 
mat.shader = 'defaultLit'
render.scene.add_geometry("sphere1", sphere, mat)
render.setup_camera(45, [0, 0, 0], [0, 0, 5.0], [0, 1, 0])

# setup background
render.scene.set_background([1.0, 1.0, 1.0, 1.0])
####################### Solution 1 #######################
cg_settings = rendering.ColorGrading(rendering.ColorGrading.Quality.ULTRA, rendering.ColorGrading.ToneMapping.LINEAR)
render.scene.view.set_color_grading(cg_settings)
####################### Solution 2 #######################
# render.scene.view.set_post_processing(False)

# render
cimg = render.render_to_image()
plt.imshow(cimg)
plt.show()

Environment

Python: 3.8.15 Open3D: 0.16.0 OS: Ubuntu 20.04

Problem with Solution 1

With solution 1, the background color is not [255, 255, 255]. Instead, it is about 253~254 (image below). I can get a [255, 255, 255] background by hacking: render.scene.set_background([10, 10, 10, 1.0]). But I just wonder whether it is possible to get 255 from render.scene.set_background([1.0, 1.0, 1.0, 1.0])

Problem with Solution 2

Solution 2 does give a [255, 255, 255] white background, however, the color is darker compared to solution 1. Is it possible to get the similar visual effects (lighter colors) with solution 1, while keeping a [255, 255, 255] background?