Dataset_Synthesizer
Dataset_Synthesizer copied to clipboard
NVIDIA
Bad class segmentation and depth images
It seems to be happening when I change the captured image size from a square image to a 1920x1080 image.
I have quite the same problem when I increase the rotation speed of the camera. The image size is 640x480. The segmentation image, depth image and the meta data are matched, but not with the rgb image.
@wightwhale what feature extractor capture the first image in that set of 4? I think you need to update the camera intrinsics when you change the captured image size. Can you screenshot your camera settings so I can see what it is set to?
The first image is depth, the last is class segmentation.
I'm trying to copy camera the settings I have from a camera we're using. If I set it to a square image like 512x512 I don't get the same issue.
[image]
width 1920
height 1080
[narrow_stereo]
camera matrix 1489.498798 0.000000 981.494096 0.000000 1494.567737 558.213286 0.000000 0.000000 1.000000
distortion 0.039849 -0.097778 0.000985 0.003079 0.000000
rectification 1.000000 0.000000 0.000000 0.000000 1.000000 0.000000 0.000000 0.000000 1.000000
projection 1488.258911 0.000000 988.192170 0.000000 0.000000 1503.269653 558.937259 0.000000 0.000000 0.000000 1.000000 0.000000
The camera settings seem fine. It look like there are several problems here:
- The captured depth is bad: it look like it have double vision. It shouldn't look like that.
- There are lag between the frame: look like the rgb feature extractor is capture in different timestamp than the other.
I can't repro both of these problems in my machine. I think it may caused by some hitches on your machine. Do you have this problem with captured image size: 1280x720? May be the size of the captured images are too big for your machine to handle.
It does work for me at 1280x720. Guess the fix for this is a faster computer?
Faster PC is always better right ?! :) The room scene is pretty complicated with a lot of 3d objects and realistic lighting setup. You can try your camera setup in the default map with more simple setup to see if it work there with 1080p resolution. Also from the depth image, I think it may caused by you hard disk's speed. Are you running the NDDS and export the images on a SSD?
I might need to trade out this laptop for a desktop. I do have a SSD, but I did turn up some of the graphic options before I ran the scenario so that could also be a cause.
I had a similar problem (see https://github.com/NVIDIA/Dataset_Synthesizer/issues/15 ), and while I never tracked down the actual cause of the bug, I think it was related to something going wrong with a merge from v1.1 to v1.2. I was able to resolve the problem by downloading a fresh copy of v1.2 and starting from scratch. If you started on your project before v1.2 was committed, and merged the changes with yours, you may be running into the same issue (at least as far as the 1st image goes). I didn't notice any sync issues, but then again, I didn't dig to deeply once I noticed the images rendering incorrectly.
In version 1.2, we upgraded the code to work with UE4 4.22 version. If you just pull the code and still using the old engine version UE4 4.21 then it would cause the problem.
I tried again with a computer having a nicer video card and the issue persists. How is the wait happening on image capture? Maybe it needs additional processing time.
@wightwhale: This sure seem to be a weird issue. If you can give me your .umap I can try to repro this issue on my side again.
The first image is depth, the last is class segmentation.
I'm trying to copy camera the settings I have from a camera we're using. If I set it to a square image like 512x512 I don't get the same issue.
[image]
width 1920
height 1080
[narrow_stereo]
camera matrix 1489.498798 0.000000 981.494096 0.000000 1494.567737 558.213286 0.000000 0.000000 1.000000
distortion 0.039849 -0.097778 0.000985 0.003079 0.000000
rectification 1.000000 0.000000 0.000000 0.000000 1.000000 0.000000 0.000000 0.000000 1.000000
projection 1488.258911 0.000000 988.192170 0.000000 0.000000 1503.269653 558.937259 0.000000 0.000000 0.000000 1.000000 0.000000
does it necessary to give camera intrinsic and camera extrinsic values before saving the images. or the default values create any errors while we use the data set for object detection.
Because i am using the custom created dataset for an object detection problem using Intel Realsense D435. So ; is it necessary to give values? or is it ok with the default UE4 values?
