Open3D
Open3D copied to clipboard
Tensorboard-plugin throws Error when selecting RGB-shader with custom vertex features
Checklist
- [X] I have searched for similar issues.
- [X] For Python issues, I have tested with the latest development wheel.
- [X] I have checked the release documentation and the latest documentation (for
mainbranch).
Describe the issue
In the tensorboard plugin, I want to visualize two different colorings ("vertex_colors") and ("vertex_modified_colors") of one object.
Visualizing the color stored in "vertex_colors" works, but visualizing a secondary color option does not.
It works to visualize one channel of the additional vertex attribute with the shader Colormap (Rainbow) and Colormap (Greyscale), but RGB does not work
Steps to reproduce the bug
1. Generate the tensorboard logs with the code below
2. Open tensorboard
3. As Data, select `modified_colors`
4. Use Shader `RGB`
#####################################################################################
import open3d as o3d
import numpy as np
# Monkey-patch torch.utils.tensorboard.SummaryWriter
from open3d.visualization.tensorboard_plugin import summary
# Utility function to convert Open3D geometry to a dictionary format
from open3d.visualization.tensorboard_plugin.util import to_dict_batch
from torch.utils.tensorboard import SummaryWriter
ply_point_cloud = o3d.data.PLYPointCloud()
pcd = o3d.io.read_point_cloud(ply_point_cloud.path)
logdir = "demo_logs/pytorch/small_scale"
writer = SummaryWriter(logdir)
pcd_dict = to_dict_batch([pcd])
# ADD ADDITIONAL VERTEX ATTRIBUTE
pcd_dict["vertex_modified_colors"] = np.copy(pcd_dict["vertex_colors"])
writer.add_3d('obj', pcd_dict, step=1)
writer.add_3d('obj', pcd_dict, step=2)
Error message
Open3D WARNING] Tensor point cloud colors must have DType of Float32 not UInt8. Converting. [Open3D INFO] OnDataChannelMessage: {"messageId":3,"class_name":"tensorboard/update_rendering","window_uid_list":["window_1"],"tag":"obj","render_state":{"property":"modified_colors","index":"0","shader":"defaultUnlit","colormap":{},"range":[0,255]},"updated":["shader"]}. Error processing message: TypeError: eq(): incompatible function arguments. The following argument types are supported: 1. (self: open3d.cpu.pybind.core.Dtype, arg0: open3d.cpu.pybind.core.Dtype) -> bool
Invoked with: UInt8, <class 'numpy.float32'>
At: /home/manuel/anaconda3/lib/python3.11/site-packages/open3d/visualization/tensorboard_plugin/util.py(407): _normalize /home/manuel/anaconda3/lib/python3.11/site-packages/open3d/visualization/tensorboard_plugin/util.py(670): apply /home/manuel/anaconda3/lib/python3.11/site-packages/open3d/visualization/tensorboard_plugin/plugin.py(625): _update_rendering
Expected behavior
The expectation is that a colored point cloud should be shown. Instead, it crashes
Open3D, Python and System information
- Operating system: Ubuntu 22.04
- Python version: 3.11.7 (main, Dec 15 2023, 18:12:31) [GCC 11.2.0]
- Open3D version: 0.18.0 ( also tested on 0.18.0+553ca86)
- System architecture: x86
- Is this a remote workstation?: no
- How did you install Open3D?: pip
Additional information
No response