lightning-bolts
lightning-bolts copied to clipboard
Callback for Plotting Bounding Boxes, Segmentation Masks.
🚀 Feature
The next release of torchvision 0.9.0 will pack a few plotting utilities that work with detection and segmentation models.
E.g. draw_bounding_boxes
, draw_segmentation_masks
,
We will have merged detection models from torchvision, (FRCNN and RetinaNet (soon)) into bolts. So this can be a nice time to implement a callback.
Motivation
It would be really nice to a Callback that allows the users to log their predictions to TensorBoard, etc.
Pitch
I'm not sure how callbacks work (will dive into it) but something like confused logit callback can work I guess (thanks @akihironitta).
Detection models in their validation mode output the predicted boxes, which are really good to visualize.
Maybe we can do something on_validation_step_end
?
Alternatives
I'm not sure if we would leave it to the user for such implementation, but I think it is nice to have.
Additional context
~Wait till next release fo torchvision, these features are still not released~ Both the features are released since torchvision 0.9 they should be available.
hi,
I wanted to know why is the draw_bounding_boxes
not using cv2
import cv2 xmin,ymin,xmax,ymax = 317,265,556,342 x = xmin y = ymin w = xmax - xmin h = ymax - ymin crop = img[y:y+h,x:x+w] plt.imshow(crop) plt.imshow(cv2.rectangle(img,(x,y),(x+w,y+h),(200,0,0),2))
Example notebook https://github.com/Programmer-RD-AI/Microcontroller-Detection-Learning-Detectron2/blob/main/00.ipynb
I just wanted to know I am trying to do this issue.
With best regards, Ranuga
Hi @Programmer-RD-AI draw_bounding_boxes
and draw_segmentation_masks
or in fact all the utilities written in torchvision.utils
do not use cv2
The major reason is to avoid an extra opencv dependency. PIL works well out of box, which is a needed dependency for torchvision. Also the utils API works really well with the models provided in torchvision.models
A simple example for them is available over torchvision gallery.
https://pytorch.org/vision/stable/auto_examples/plot_visualization_utils.html#sphx-glr-auto-examples-plot-visualization-utils-py
Your above code can be rewritten very easily with torchvision utils
import torch
from torchvision.io import read_image
img = read_image("pathto_file.jpg")
xmin, ymin, xmax, ymax = 317, 265, 556, 342
bbox_tensor = torch.tensor([xmin, ymin, xmax, ymax], dtype=torch.float)
res = draw_bounding_boxes(img, bbox_tensor, colors="red", label=["Image"], width=3)
draw_bounding_boxes
is more robust, it can directly draw boxes on tensor, add a label, fill with same semi transparent color (use fill=True). You can also adjust the width of the box.
Read more about utilities in the docs.
https://pytorch.org/vision/stable/utils.html#torchvision.utils.draw_bounding_boxes
ok thank you @oke-aditya
I will check the resources.
With best regards, Ranuga
Curious about the progress on this? Is there a built-in way in PyTorch Lighting, or Flash or Bolts to visualize bboxes from model predictions?