BlenderProc
BlenderProc copied to clipboard
DLR-RM
Integrate pass rendering for semantic segmentation
Fixes #686
This removes the need for calling the rendering function twice, going from this:
# render the whole pipeline
data = bproc.renderer.render()
# Render segmentation masks (per class and per instance)
data.update(bproc.renderer.render_segmap(map_by=["class", "instance", "name"]))
To this:
bproc.renderer.enable_semantic_segmentation(map_by=["class", "instance", "name"])
# render the whole pipeline
data = bproc.renderer.render()
TODO:
- [x] make sure it works in all settings
- [x] add proper documentation
If we kill the config yaml files then yes. I will not port the old config yaml modules to this new renderer.
BlenderProcHelper result for run 397:
Done with:
Quicker Examples: [
amass_human_poses 1s, amass_human_poses_python 3s, auto_shading 2s, auto_shading_python 5s, basic 4s, basic_python 7s, blenderkit 3s, blenderkit_python 9s, bop_object_on_surface_sampling, bop_object_on_surface_sampling_python 2s, bop_object_physics_positioning, bop_object_physics_positioning_python 2s, bop_object_pose_sampling_0 14s, bop_object_pose_sampling_0_python 2s, bop_object_pose_sampling_1, bop_object_pose_sampling_1_python 2s, bop_scene_replication, bop_scene_replication_python 2s, camera_depth_of_field 2s, camera_depth_of_field_python 10s, camera_object_pose 5s, camera_object_pose_python 8s, camera_random_trajectories_python, camera_sampling 2s, camera_sampling_python, coco_annotations 3s, coco_annotations_python 7s, diffuse_color_image 3s, diffuse_color_image_python 10s, dust 2s, dust_python, entity_displacement_modifier 5s, entity_displacement_modifier_python 7s, entity_manipulation 4s, entity_manipulation_python 5s, front_3d 6s, front_3d_python, front_3d_with_improved_mat 8s, front_3d_with_improved_mat_python, git_log 1s, haven 2s, haven_python 12s, ikea 1s, ikea_python 9s, lens_distortion 2s, lens_distortion_python 10s, lens_distortion_python 6s, light_sampling 5s, light_sampling_python 7s, material_manipulation 2s, material_manipulation_python 5s, material_randomizer 2s, material_randomizer_python 7s, matterport3d_python, motion_blur_rolling_shutter 9s, motion_blur_rolling_shutter 8s, motion_blur_rolling_shutter_python 13s, multi_render_python 2s, nocs_python 9s, object_pose_sampling 2s, object_pose_sampling_python 5s, on_surface_object_sampling, on_surface_object_sampling_python 5s, optical_flow 10s, physics_convex_decomposition, physics_convex_decomposition_python, physics_positioning 9s, physics_positioning_python 12s, pix3d 1s, pix3d_python 8s, random_backgrounds 3s, random_backgrounds_python 12s, random_room_constructor 9s, random_room_constructor_python, replica, replica_python, rock_essentials, scenenet 3s, scenenet_python, scenenet_with_cctextures, scenenet_with_cctextures_python, semantic_segmentation 3s, semantic_segmentation_python 7s, shapenet 2s, shapenet_python 11s, shapenet_with_scenenet 4s, shapenet_with_scenenet_python, shapenet_with_suncg 11s, shapenet_with_suncg_python, stereo_matching, stereo_matching_python, suncg_basic, suncg_with_cam_sampling 11s, suncg_with_cam_sampling_python, suncg_with_improved_mat 8s, suncg_with_improved_mat_python, suncg_with_object_replacer, suncg_with_object_replacer_python, tests_python 2s, urdf_loading_and_manipulation_python]
Done with 103 of 103 tests.
There were errors:
The output of
coco_annotations:config_yaml:coco_data_coco_annotations_json has changed:
| compare.json | newly_generated.json | difference.json |
|---|---|---|
The output of coco_annotations:main_python:coco_data_coco_annotations_json has changed:
| compare.json | newly_generated.json | difference.json |
|---|---|---|
The output for comparing for coco_annotations:main_python does not exist:
${REC_DATA_STORAGE}/blenderproc/BlenderProcCmpData/coco_annotations/main_python/coco_data/images/000002.jpg
└── coco_data
├── coco_annotations.json
└── images
├── 000000.jpg
├── 000001.jpg
├── 000002.jpg
└── 000003.jpg
2 directories, 5 files
For semantic_segmentation:main_python:category_id_segmaps: This category_id_segmaps key is new and was not there before
For semantic_segmentation:main_python:class_segmaps: This class_segmaps key is not there anymore
For semantic_segmentation:main_python:category_id_segmaps: This category_id_segmaps key is new and was not there before
For semantic_segmentation:main_python:class_segmaps: This class_segmaps key is not there anymore
The output of random_backgrounds:config_yaml:coco_data_coco_annotations_json has changed:
| compare.json | newly_generated.json | difference.json |
|---|---|---|
The output for comparing for random_backgrounds:config_yaml does not exist:
${REC_DATA_STORAGE}/blenderproc/BlenderProcCmpData/random_backgrounds/config_yaml/coco_data/images/rgb_0001.png
└── coco_data
├── coco_annotations.json
└── images
├── rgb_0000.png
├── rgb_0001.png
├── rgb_0002.png
├── rgb_0003.png
├── rgb_0004.png
├── rgb_0005.png
├── rgb_0006.png
├── rgb_0007.png
└── rgb_0008.png
2 directories, 10 files
The output of random_backgrounds:main_python:coco_data_coco_annotations_json has changed:
| compare.json | newly_generated.json | difference.json |
|---|---|---|
The output for comparing for random_backgrounds:main_python does not exist:
${REC_DATA_STORAGE}/blenderproc/BlenderProcCmpData/random_backgrounds/main_python/coco_data/images/000005.png
└── coco_data
├── coco_annotations.json
└── images
├── 000000.png
├── 000001.png
├── 000002.png
├── 000003.png
├── 000004.png
├── 000005.png
├── 000006.png
├── 000007.png
├── 000008.png
├── 000009.png
├── 000010.png
├── 000011.png
├── 000012.png
├── 000013.png
├── 000014.png
├── 000015.png
├── 000016.png
├── 000017.png
├── 000018.png
├── 000019.png
├── 000020.png
├── 000021.png
├── 000022.png
├── 000023.png
├── 000024.png
├── 000025.png
├── 000026.png
├── 000027.png
├── 000028.png
├── 000029.png
├── 000030.png
├── 000031.png
├── 000032.png
├── 000033.png
├── 000034.png
├── 000035.png
├── 000036.png
├── 000037.png
├── 000038.png
├── 000039.png
├── 000040.png
├── 000041.png
├── 000042.png
├── 000043.png
├── 000044.png
├── 000045.png
├── 000046.png
├── 000047.png
├── 000048.png
├── 000049.png
├── 000050.png
├── 000051.png
├── 000052.png
├── 000053.png
├── 000054.png
├── 000055.png
├── 000056.png
├── 000057.png
├── 000058.png
└── 000059.png
2 directories, 61 files
The output could not be compared as no files have been generated for amass_human_poses:config_yaml!
The output could not be compared as no files have been generated for amass_human_poses:main_python!
The output of random_room_constructor:config_yaml:colors has changed: Diff: 0.2050216645002365, pixel diff: 0.9631131490071615
| Compare image | Newly generated image |
|---|
The output of random_room_constructor:config_yaml:depth has changed: Diff: 0.04018905758857727, pixel diff: 0.5409812927246094
| Compare image | Newly generated image |
|---|
The output could not be compared as no files have been generated for bop_object_physics_positioning:main_python!
The output could not be compared as no files have been generated for bop_object_on_surface_sampling:main_python!
The output could not be compared as no files have been generated for bop_scene_replication:main_python!
For suncg_basic:main_python:category_id_segmaps: This category_id_segmaps key is new and was not there before
For suncg_basic:main_python:instance_attribute_maps: This instance_attribute_maps key is new and was not there before
For suncg_basic:main_python:class_segmaps: This class_segmaps key is not there anymore
For suncg_basic:main_python:category_id_segmaps: This category_id_segmaps key is new and was not there before
For suncg_basic:main_python:instance_attribute_maps: This instance_attribute_maps key is new and was not there before
For suncg_basic:main_python:class_segmaps: This class_segmaps key is not there anymore
The output could not be compared as no files have been generated for multi_render:main_python!
Logs of the error examples:
- coco annotations
- coco annotations python
- semantic segmentation python
- random backgrounds
- random backgrounds python
- amass human poses
- amass human poses python
- random room constructor
- bop object physics positioning python
- bop object on surface sampling python
- bop scene replication python
- suncg basic python
- multi render python
There are config files, which have not been tested:
- examples/advanced/stereo_matching_with_projector/config.yaml
- examples/advanced/gif_animation/main.py
- examples/advanced/stereo_matching_with_projector/main.py
- examples/advanced/kinect_azure_noise/main.py
- examples/datasets/front_3d_object_sampling/main.py
The tests are done. Check the log:
${INTERNAL_HOME_FOLDER}/Maximilian_Denninger/workspace/github-actions-runner/_work/BlenderProc/BlenderProc/blenderproc/scripts/visHdf5Files.py:78: RuntimeWarning: More than 20 figures have been opened. Figures created through the pyplot interface (`matplotlib.pyplot.figure`) are retained until explicitly closed and may consume too much memory. (To control this warning, see the rcParam `figure.max_open_warning`).
plt.figure()
Matplotlib is building the font cache; this may take a moment.
${INTERNAL_HOME_FOLDER}/Maximilian_Denninger/workspace/github-actions-runner/_work/BlenderProc/BlenderProc/blenderproc/scripts/visHdf5Files.py:78: RuntimeWarning: More than 20 figures have been opened. Figures created through the pyplot interface (`matplotlib.pyplot.figure`) are retained until explicitly closed and may consume too much memory. (To control this warning, see the rcParam `figure.max_open_warning`).
plt.figure()
Matplotlib is building the font cache; this may take a moment.
${INTERNAL_HOME_FOLDER}/Maximilian_Denninger/workspace/github-actions-runner/_work/BlenderProc/BlenderProc/blenderproc/scripts/visHdf5Files.py:78: RuntimeWarning: More than 20 figures have been opened. Figures created through the pyplot interface (`matplotlib.pyplot.figure`) are retained until explicitly closed and may consume too much memory. (To control this warning, see the rcParam `figure.max_open_warning`).
plt.figure()
Warning: The data with key 'distance' has more than one channel which would not allow using a jet color map. Therefore only the first channel is visualized.
Warning: The data with key 'distance' has more than one channel which would not allow using a jet color map. Therefore only the first channel is visualized.
Matplotlib is building the font cache; this may take a moment.
Warning: The data with key 'distance' has more than one channel which would not allow using a jet color map. Therefore only the first channel is visualized.
Warning: The data with key 'distance' has more than one channel which would not allow using a jet color map. Therefore only the first channel is visualized.
Warning: The data with key 'distance' has more than one channel which would not allow using a jet color map. Therefore only the first channel is visualized.
Warning: The data with key 'distance' has more than one channel which would not allow using a jet color map. Therefore only the first channel is visualized.
Warning: The data with key 'distance' has more than one channel which would not allow using a jet color map. Therefore only the first channel is visualized.
Warning: The data with key 'distance' has more than one channel which would not allow using a jet color map. Therefore only the first channel is visualized.
Warning: The data with key 'distance' has more than one channel which would not allow using a jet color map. Therefore only the first channel is visualized.
Warning: The data with key 'distance' has more than one channel which would not allow using a jet color map. Therefore only the first channel is visualized.
${INTERNAL_HOME_FOLDER}/Maximilian_Denninger/workspace/github-actions-runner/_work/BlenderProc/BlenderProc/blenderproc/scripts/visHdf5Files.py:78: RuntimeWarning: More than 20 figures have been opened. Figures created through the pyplot interface (`matplotlib.pyplot.figure`) are retained until explicitly closed and may consume too much memory. (To control this warning, see the rcParam `figure.max_open_warning`).
plt.figure()
Last update: 09/22/2022, 10:33:05