Code for scene graph visualization

[iamsusiep]

❓ Questions and Help

Is there code for scene graph visualization after running inference/evaluation?

[KaihuaTang]

My visualization codes are written in several independent jupyter notes, and not included in the main project by now. I will consider releasing them in the future after I organize all the codes.

[Iceland-Leo]

@KaihuaTang Have you organized the code for the visualization section? This is what I need urgently at present. Even if it is still in jupyter format, can it be made public? Thank you！

[KaihuaTang]

@KaihuaTang Have you organized the code for the visualization section? This is what I need urgently at present. Even if it is still in jupyter format, can it be made public? Thank you！

Check https://github.com/KaihuaTang/Scene-Graph-Benchmark.pytorch/blob/master/visualization/1.visualize_PredCls_and_SGCls.ipynb

bbox with label & index are drawn on the picture. ground-truth relations and predicted relations are printed as a list. Since the predicted relations are pretty dense, you need to set a threshold to show the top-K. Then I drew the graphs through ppt for the best visual effects.

[Iceland-Leo]

@KaihuaTang Thank you very much! One more question, when I start training, the glove will be download. But an error ocurred. File "/mnt/disk/cv/Liucheng/pytorchCode/Scene-Graph-Benchmark.pytorch-master/maskrcnn_benchmark/modeling/roi_heads/relation_head/roi_relation_predictors.py", line 459, in __init__ self.context_layer = VCTreeLSTMContext(config, obj_classes, rel_classes, statistics, in_channels) File "/mnt/disk/cv/Liucheng/pytorchCode/Scene-Graph-Benchmark.pytorch-master/maskrcnn_benchmark/modeling/roi_heads/relation_head/model_vctree.py", line 93, in __init__ obj_embed_vecs = obj_edge_vectors(self.obj_classes, wv_dir=self.cfg.GLOVE_DIR, wv_dim=self.embed_dim) File "/mnt/disk/cv/Liucheng/pytorchCode/Scene-Graph-Benchmark.pytorch-master/maskrcnn_benchmark/modeling/roi_heads/relation_head/utils_motifs.py", line 152, in obj_edge_vectors wv_dict, wv_arr, wv_size = load_word_vectors(wv_dir, wv_type, wv_dim) File "/mnt/disk/cv/Liucheng/pytorchCode/Scene-Graph-Benchmark.pytorch-master/maskrcnn_benchmark/modeling/roi_heads/relation_head/utils_motifs.py", line 205, in load_word_vectors with zipfile.ZipFile(fname, "r") as zf: File "/home/cilab-cv/miniconda3/envs/sg/lib/python3.6/zipfile.py", line 1131, in __init__ self._RealGetContents() File "/home/cilab-cv/miniconda3/envs/sg/lib/python3.6/zipfile.py", line 1198, in _RealGetContents raise BadZipFile("File is not a zip file") zipfile.BadZipFile: File is not a zip file

[KaihuaTang]

I don't know. I‘ve never seen this bug before.

[JincanDeng]

@KaihuaTang Can you release the visualization code for sgdet results?

[KaihuaTang]

@KaihuaTang Can you release the visualization code for sgdet results?

It seems like I may lost this part of codes. The main difference between sgdet visualization and others is a bbox selection function (with threshold), otherwise there would be too many bboxes. I'm currently working on another project, maybe you can write your own.

[navidre]

@KaihuaTang Thanks for the great and tidy code.

@JincanDeng A PR is opened to add the SGDet visualization. Not many changes from the SGCls notebook.

[runzeer]

@navidre Thanks for your visualization for the SGDets code. Could you tell me how to get to your visualization results? Especially for the pred scores? The Numerical values seem very small. Why are your results are so big?

[runzeer]

@navidre if connenient, could you share the config file for test?

[folkaholic]

@KaihuaTang Thank you very much! One more question, when I start training, the glove will be download. But an error ocurred. File "/mnt/disk/cv/Liucheng/pytorchCode/Scene-Graph-Benchmark.pytorch-master/maskrcnn_benchmark/modeling/roi_heads/relation_head/roi_relation_predictors.py", line 459, in __init__ self.context_layer = VCTreeLSTMContext(config, obj_classes, rel_classes, statistics, in_channels) File "/mnt/disk/cv/Liucheng/pytorchCode/Scene-Graph-Benchmark.pytorch-master/maskrcnn_benchmark/modeling/roi_heads/relation_head/model_vctree.py", line 93, in __init__ obj_embed_vecs = obj_edge_vectors(self.obj_classes, wv_dir=self.cfg.GLOVE_DIR, wv_dim=self.embed_dim) File "/mnt/disk/cv/Liucheng/pytorchCode/Scene-Graph-Benchmark.pytorch-master/maskrcnn_benchmark/modeling/roi_heads/relation_head/utils_motifs.py", line 152, in obj_edge_vectors wv_dict, wv_arr, wv_size = load_word_vectors(wv_dir, wv_type, wv_dim) File "/mnt/disk/cv/Liucheng/pytorchCode/Scene-Graph-Benchmark.pytorch-master/maskrcnn_benchmark/modeling/roi_heads/relation_head/utils_motifs.py", line 205, in load_word_vectors with zipfile.ZipFile(fname, "r") as zf: File "/home/cilab-cv/miniconda3/envs/sg/lib/python3.6/zipfile.py", line 1131, in __init__ self._RealGetContents() File "/home/cilab-cv/miniconda3/envs/sg/lib/python3.6/zipfile.py", line 1198, in _RealGetContents raise BadZipFile("File is not a zip file") zipfile.BadZipFile: File is not a zip file

Did you solve this problem？ I encountered the same problem. It is very strange that I downloaded glove.6B.zip in the glove folder, but every time I train, it will re-download a file called glove.6B.200d. Please help me, thank you!

[KaihuaTang]

@KaihuaTang Thank you very much! One more question, when I start training, the glove will be download. But an error ocurred. File "/mnt/disk/cv/Liucheng/pytorchCode/Scene-Graph-Benchmark.pytorch-master/maskrcnn_benchmark/modeling/roi_heads/relation_head/roi_relation_predictors.py", line 459, in __init__ self.context_layer = VCTreeLSTMContext(config, obj_classes, rel_classes, statistics, in_channels) File "/mnt/disk/cv/Liucheng/pytorchCode/Scene-Graph-Benchmark.pytorch-master/maskrcnn_benchmark/modeling/roi_heads/relation_head/model_vctree.py", line 93, in __init__ obj_embed_vecs = obj_edge_vectors(self.obj_classes, wv_dir=self.cfg.GLOVE_DIR, wv_dim=self.embed_dim) File "/mnt/disk/cv/Liucheng/pytorchCode/Scene-Graph-Benchmark.pytorch-master/maskrcnn_benchmark/modeling/roi_heads/relation_head/utils_motifs.py", line 152, in obj_edge_vectors wv_dict, wv_arr, wv_size = load_word_vectors(wv_dir, wv_type, wv_dim) File "/mnt/disk/cv/Liucheng/pytorchCode/Scene-Graph-Benchmark.pytorch-master/maskrcnn_benchmark/modeling/roi_heads/relation_head/utils_motifs.py", line 205, in load_word_vectors with zipfile.ZipFile(fname, "r") as zf: File "/home/cilab-cv/miniconda3/envs/sg/lib/python3.6/zipfile.py", line 1131, in __init__ self._RealGetContents() File "/home/cilab-cv/miniconda3/envs/sg/lib/python3.6/zipfile.py", line 1198, in _RealGetContents raise BadZipFile("File is not a zip file") zipfile.BadZipFile: File is not a zip file

Did you solve this problem？ I encountered the same problem. It is very strange that I downloaded glove.6B.zip in the glove folder, but every time I train, it will re-download a file called glove.6B.200d. Please help me, thank you!

I'm not sure. Maybe changing the path of the customized GLOVE_DIR in the command?

[Iceland-Leo]

You can unzip the glove.6b.zip, then train the model.

发自我的iPhone

------------------ Original ------------------ From: Kaihua Tang <[email protected]> Date: Mon,Apr 13,2020 10:20 PM To: KaihuaTang/Scene-Graph-Benchmark.pytorch <[email protected]> Cc: Liucheng <[email protected]>, Comment <[email protected]> Subject: Re: [KaihuaTang/Scene-Graph-Benchmark.pytorch] Code for scene graph visualization (#2)

@KaihuaTang Thank you very much! One more question, when I start training, the glove will be download. But an error ocurred. File "/mnt/disk/cv/Liucheng/pytorchCode/Scene-Graph-Benchmark.pytorch-master/maskrcnn_benchmark/modeling/roi_heads/relation_head/roi_relation_predictors.py", line 459, in init self.context_layer = VCTreeLSTMContext(config, obj_classes, rel_classes, statistics, in_channels) File "/mnt/disk/cv/Liucheng/pytorchCode/Scene-Graph-Benchmark.pytorch-master/maskrcnn_benchmark/modeling/roi_heads/relation_head/model_vctree.py", line 93, in init obj_embed_vecs = obj_edge_vectors(self.obj_classes, wv_dir=self.cfg.GLOVE_DIR, wv_dim=self.embed_dim) File "/mnt/disk/cv/Liucheng/pytorchCode/Scene-Graph-Benchmark.pytorch-master/maskrcnn_benchmark/modeling/roi_heads/relation_head/utils_motifs.py", line 152, in obj_edge_vectors wv_dict, wv_arr, wv_size = load_word_vectors(wv_dir, wv_type, wv_dim) File "/mnt/disk/cv/Liucheng/pytorchCode/Scene-Graph-Benchmark.pytorch-master/maskrcnn_benchmark/modeling/roi_heads/relation_head/utils_motifs.py", line 205, in load_word_vectors with zipfile.ZipFile(fname, "r") as zf: File "/home/cilab-cv/miniconda3/envs/sg/lib/python3.6/zipfile.py", line 1131, in init self._RealGetContents() File "/home/cilab-cv/miniconda3/envs/sg/lib/python3.6/zipfile.py", line 1198, in _RealGetContents raise BadZipFile("File is not a zip file") zipfile.BadZipFile: File is not a zip file

Did you solve this problem？ I encountered the same problem. It is very strange that I downloaded glove.6B.zip in the glove folder, but every time I train, it will re-download a file called glove.6B.200d. Please help me, thank you!

I'm not sure. Maybe changing the path of the customized GLOVE_DIR in the command?

— You are receiving this because you commented. Reply to this email directly, view it on GitHub, or unsubscribe.

[folkaholic]

@KaihuaTang Thank you very much! One more question, when I start training, the glove will be download. But an error ocurred. File "/mnt/disk/cv/Liucheng/pytorchCode/Scene-Graph-Benchmark.pytorch-master/maskrcnn_benchmark/modeling/roi_heads/relation_head/roi_relation_predictors.py", line 459, in __init__ self.context_layer = VCTreeLSTMContext(config, obj_classes, rel_classes, statistics, in_channels) File "/mnt/disk/cv/Liucheng/pytorchCode/Scene-Graph-Benchmark.pytorch-master/maskrcnn_benchmark/modeling/roi_heads/relation_head/model_vctree.py", line 93, in __init__ obj_embed_vecs = obj_edge_vectors(self.obj_classes, wv_dir=self.cfg.GLOVE_DIR, wv_dim=self.embed_dim) File "/mnt/disk/cv/Liucheng/pytorchCode/Scene-Graph-Benchmark.pytorch-master/maskrcnn_benchmark/modeling/roi_heads/relation_head/utils_motifs.py", line 152, in obj_edge_vectors wv_dict, wv_arr, wv_size = load_word_vectors(wv_dir, wv_type, wv_dim) File "/mnt/disk/cv/Liucheng/pytorchCode/Scene-Graph-Benchmark.pytorch-master/maskrcnn_benchmark/modeling/roi_heads/relation_head/utils_motifs.py", line 205, in load_word_vectors with zipfile.ZipFile(fname, "r") as zf: File "/home/cilab-cv/miniconda3/envs/sg/lib/python3.6/zipfile.py", line 1131, in __init__ self._RealGetContents() File "/home/cilab-cv/miniconda3/envs/sg/lib/python3.6/zipfile.py", line 1198, in _RealGetContents raise BadZipFile("File is not a zip file") zipfile.BadZipFile: File is not a zip file

Did you solve this problem？ I encountered the same problem. It is very strange that I downloaded glove.6B.zip in the glove folder, but every time I train, it will re-download a file called glove.6B.200d. Please help me, thank you!

I'm not sure. Maybe changing the path of the customized GLOVE_DIR in the command?

Thank you for your prompt reply！My problem may solved by @Iceland-Leo .Because the zip file I downloaded have problem in unzipping, the system will re-download the unzip file of glove.6B.zip everytime I train the model.

[folkaholic]

You can unzip the glove.6b.zip, then train the model. 发自我的iPhone … ------------------ Original ------------------ From: Kaihua Tang <[email protected]> Date: Mon,Apr 13,2020 10:20 PM To: KaihuaTang/Scene-Graph-Benchmark.pytorch <[email protected]> Cc: Liucheng <[email protected]>, Comment <[email protected]> Subject: Re: [KaihuaTang/Scene-Graph-Benchmark.pytorch] Code for scene graph visualization (#2) @KaihuaTang Thank you very much! One more question, when I start training, the glove will be download. But an error ocurred. File "/mnt/disk/cv/Liucheng/pytorchCode/Scene-Graph-Benchmark.pytorch-master/maskrcnn_benchmark/modeling/roi_heads/relation_head/roi_relation_predictors.py", line 459, in init self.context_layer = VCTreeLSTMContext(config, obj_classes, rel_classes, statistics, in_channels) File "/mnt/disk/cv/Liucheng/pytorchCode/Scene-Graph-Benchmark.pytorch-master/maskrcnn_benchmark/modeling/roi_heads/relation_head/model_vctree.py", line 93, in init obj_embed_vecs = obj_edge_vectors(self.obj_classes, wv_dir=self.cfg.GLOVE_DIR, wv_dim=self.embed_dim) File "/mnt/disk/cv/Liucheng/pytorchCode/Scene-Graph-Benchmark.pytorch-master/maskrcnn_benchmark/modeling/roi_heads/relation_head/utils_motifs.py", line 152, in obj_edge_vectors wv_dict, wv_arr, wv_size = load_word_vectors(wv_dir, wv_type, wv_dim) File "/mnt/disk/cv/Liucheng/pytorchCode/Scene-Graph-Benchmark.pytorch-master/maskrcnn_benchmark/modeling/roi_heads/relation_head/utils_motifs.py", line 205, in load_word_vectors with zipfile.ZipFile(fname, "r") as zf: File "/home/cilab-cv/miniconda3/envs/sg/lib/python3.6/zipfile.py", line 1131, in init self._RealGetContents() File "/home/cilab-cv/miniconda3/envs/sg/lib/python3.6/zipfile.py", line 1198, in _RealGetContents raise BadZipFile("File is not a zip file") zipfile.BadZipFile: File is not a zip file Did you solve this problem？ I encountered the same problem. It is very strange that I downloaded glove.6B.zip in the glove folder, but every time I train, it will re-download a file called glove.6B.200d. Please help me, thank you! I'm not sure. Maybe changing the path of the customized GLOVE_DIR in the command? — You are receiving this because you commented. Reply to this email directly, view it on GitHub, or unsubscribe.

Thank you for your prompt reply！Problem solved!

[dami23]

thank you for the share. A question about the visualize code, what is the definition of 'dispaly' in draw_iamge? Thank you!