[Bug Report] Adding call for CV2 imshow will freeze entire Isaac-Sim

[ahanjaya]

Describe the bug

The bug is about calling the OpenCV window to render the camera image from the robot. The entire simulation will stuck on startup.

Steps to reproduce

def _debug_vis_callback(self, event):
     cv2.imshow("Robot Depth Frame", self.grid_depth_img)
     cv2.waitKey(1)

Please try to provide a minimal example to reproduce the bug. Error messages and stack traces are also helpful.

System Info

Describe the characteristics of your environment:

• Commit: 6451d235785780167f80c14bb9d635d20aff9b29
• Isaac Sim Version: 4.0.0-rc.21+4.0.13872.3e3cb0c9.gl
• OS: Ubuntu 22.04
• GPU: RTX 4090
• CUDA: 12.3
• GPU Driver: 545.23.08

Additional context

Previously this bug never occurred, I could easily stream the OpenCV result with the same code

Checklist

• [x] I have checked that there is no similar issue in the repo (required)
• [ ] I have checked that the issue is not in running Isaac Sim itself and is related to the repo

Acceptance Criteria

Add the criteria for which this task is considered done. If not known at issue creation time, you can add this once the issue is assigned.

• [ ] Criteria 1
• [ ] Criteria 2

[jsmith-bdai]

I'm not super familiar with cv2, but I think you can remove the cv2.waitKey(1) call to avoid blocking the simulation environment.

Let us know if this works for your issue!

[ahanjaya]

it still doesn't work, even by simply initializing the OpenCV window

def _set_debug_vis_impl(self, debug_vis: bool):
    if self.cfg.show_depth:
        cv2.namedWindow("Robot Depth Frame", cv2.WINDOW_NORMAL)

I did this before and had no problem the last 2 weeks, but after I rebased to the latest main branch this issue occured.

[RandomOakForest]

Our apologies we couldn't follow up in a timely manner. If you still have this issue with the latest Isaac Sim, please post a topic in the forums. Thank you.