Non-blocking visualizer getting stuck when using multiprocessor/ multithreaded application

[LHRaceIng]

Checklist

• [X] I have searched for similar issues.
• [X] For Python issues, I have tested with the latest development wheel.
• [X] I have checked the release documentation and the latest documentation (for master branch).

My Question

Hi everyone, I am trying to display a constantly updating point cloud with the non-blocking open3d visualizer. For this I am using the following code, which I copied and modified from Carla video tutorials-An in depth look at CARLA's sensors -CODE :

def add_open3d_axis(vis):
#Add a small 3D axis on Open3D Visualizer
    axis = o3d.geometry.LineSet()
    axis.points = o3d.utility.Vector3dVector(np.array([
        [0.0, 0.0, 0.0],
        [1.0, 0.0, 0.0],
        [0.0, 1.0, 0.0],
        [0.0, 0.0, 1.0]]))
    axis.lines = o3d.utility.Vector2iVector(np.array([
        [0, 1],
        [0, 2],
        [0, 3]]))
    axis.colors = o3d.utility.Vector3dVector(np.array([
        [1.0, 0.0, 0.0],
        [0.0, 1.0, 0.0],
        [0.0, 0.0, 1.0]]))
    vis.add_geometry(axis)

vis = o3d.visualization.Visualizer()
vis.create_window(
    window_name='Carla Lidar',
    width=960,
    height=540,
    left=480,
    top=270)
vis.get_render_option().background_color = [0.05, 0.05, 0.05]
vis.get_render_option().point_size = 1
vis.get_render_option().show_coordinate_frame = True
self.add_open3d_axis(vis)

start_time = time.time()
frame = 0
while time.time()-start_time<config.simulation_time:
    if not isinstance(self.data_dict['pcl_points'], type(None)):
        pcl = o3d.geometry.PointCloud()
        pcl.points = o3d.utility.Vector3dVector(self.data_dict['pcl_points'])
        pcl.colors = o3d.utility.Vector3dVector(self.data_dict['pcl_intensity'])

        if frame == 2:
            vis.add_geometry(pcl)
        vis.update_geometry(pcl)

        vis.poll_events()
        vis.update_renderer()
        # This can fix Open3D jittering issues:
        time.sleep(0.01)
        frame += 1
vis.destroy_window()

This works just fine if I run this code on a single process/ main thread. However, since my program is getting more complex I need to use multiprocessing/ multithreading. The general structure consists of two processes p1 and p2 running in parallel where p1 is creating the point cloud and p2 is used to visualize it. On p2 there are multiple threads running since it not only displays the PCL but as well image data using OpenCV. Now I run into the problem that the visualization gets stuck, only displaying the very first instance of the point cloud received in the first timestep without updating. If I substitute vis.update_geometry(pcl) with vis.add_geometry(pcl) it just adds a new point cloud every timestep (without deleting the prior one). So, I am certain that I am receiving the updated point cloud, it just fails to update the visualizer and delete the old one. I already read in other issues (#1791, #389) that using the visualizer in multithreaded/ multiprocessor applications is somewhat problematic. However, I was not able to fix my problem with the solutions mentioned there. I am not very familiar with multiprocessor applications so I am grateful for any help. Thanks in advance.