argilla
argilla copied to clipboard
[FEATURE] Image Bounding Box Annotations for vision & multimodal workflows
Hi team!
Given the awesome SpanQuestion new feature recently released, I'm tempted to ask if it's possible to have the same done for annotating regions of interests for images. It would be marvelous if there's a way to draw and label rectangle bounding boxes on an ImageField
, similar to LabelStudio's bounding boxes. I love Argilla and hope this feature can be on the roadmap!
Is your feature request related to a problem? Please describe. As LLMs are increasingly becoming multimodal and many data workflows involve a mixture of text, document, and image data types, a very common task is to highlight specific regions in an image for annotation or other downstream processing. For a specific use case, suppose an LLM is tasked with detecting objects and return a bounding box of the detected objects in JSON. It would be great to be able to take this JSON output and have humans add, edit or label the bboxes, which can either be used for finetuning multimodal LLMs or downstream tasks.
Describe the solution you'd like
Currently images are displayed using image_to_html
to encode images as html in a TextField
. We can create a ImageField
to contain an image url and other metadata (width, height, scale, offsets), and a ImageSpanQuestion
where users can create RectangleLabels
and PointLabels
stored as responses or suggestions.
Describe alternatives you've considered There isn't really a workaround to annotating region of interests without drawing on top of the image.
### Tasks