Depth file does not exist

[leesunmyung]

Hi, first of all, thank you for your great project! I got an error like below while running code with nuscenes data.

Traceback (most recent call last): File "main.py", line 240, in <module> generator.run() File "C:\Users\user\Desktop\rain-rendering\common\generator.py", line 317, in run assert os.path.exists(depth_file), "Depth file {} does not exist".format(depth_file)

AssertionError: Depth file data\source\nuscenes\samples/CAM_FRONT/n008-2018-08-30-15-16-55-0400__CAM_FRONT__1535657108262404.npy does not exist

I wonder how to obtain the depth file(.npy). I'll appreciate if you help me. Thanks. :)

[rdecharette]

Hello,

happy that our research is useful. Sadly, we haven't shared (and will likely not share) the nuScenes data as it will require too much work.

Depth data for other datasets can be downloaded here (https://github.com/astra-vision/rain-rendering?tab=readme-ov-file#data-to-generate-rain-on-kitti-cityscapes-nuscenes). Best

[leesunmyung]

Thanks for your reply!

I must render the rainy scenes with nuScenes, is there any idea to generate the depth data for nuScenes? Or could you let me know just roughly how to generate rainy-augmented scene using nuScenes?

Thanks.

[rdecharette]

Hi @leesunmyung,

rendering on nuscene is indeed possible. I'm expecting it to be a bit cumbersome, but with a couple of error-and-trial you should be able to do that. Before trying that I recommend you read all the closed issues, as some like this one have interesting insights.

To render rain on nuscenes, which is similar to generating on a custom dataset. In substance:

1. Generate the particles data. This means running the weather particle simulator. For nuscene, you may need to do some manual adustments, but you should first try by running simply the simulator, going into "SPECIAL SETTINGS" (menu 99), then "NuScenes" (menu 1000). Then start the step simulation (menu 102, then menu 1). This should generate the particles files the renderer will require (within a few hours).
2. Generate depth maps. Find a way to generate the depth maps for your corresponding camera images. Have a look at this thread.
3. One you have depth and particles, you can render rain. Prepare the config file and then run the rendering.

Once again, I expect this to be a few days process as the code is big and might lack detailed documentation, but I'll try guiding when I can. Actually, if you render proper rain on nuScene, I'll be happy to share it on our page as I got a couple of requests. Best.

[leesunmyung]

Thank you for helping me, so I successfully made depth map using MonoDepth2.

Related to this part '3. One you have depth and particles, you can render rain. Prepare the and then run the rendering.', should I make a new config file although I use nuScenes?

There already exists config/nuscenes.py so, I confused.

[leesunmyung]

Finally, I made rainy images based on nuScenes. Thanks.