Q.. component labels, workflow

[dobkeratops]

some general comments & ideas ...

(i) r.e the importance of component labels:- (a) could they accelerate training: because they narrow down feature detectors more? (b) could they give more expressive power with the curated label list, i.e. labelling components even if you dont have the labels for the object yet ("wheel / container" (.. on a wheelie-bin , or on a wheelbarrow.)

(ii) regarding the current workflow I still think components are hard to use i.e. i haven't been able to find them in search mode; it also doesnt' work well to automatically add components for the main label (because the system doesn't know which components are visible)

(iii) I still think Label-Me's hierarchy idea is extremely useful (although this is a whole extra dimension to UI, so it might be a lot of work): it lets you ground the annotations into a tree - this could sidestep the need for seperate "head/dog", "head/person" etc labels - instead what you do is label a "head", (and "paw", "tail" ..) ,and group it under an enclosing "dog" label. This could work well re. (i)(b). e.g. if you have a "wheel", "handle", "container" labels, then group them as one unit, you've described a "wheelbarrow", without needing the 'wheelbarrow' label yet

regarding UI there's the complexity of code but also user discovery. LabelMe uses drag-drop in a tree view, I think that's fairly intuitive .. but it still isn't obvious. I was trying to do it with lines to indicate parent-child links (more of a general purpose geometry editor) and I dont think would be too complex for casual users.

There's another idea which might suit your app better, which is to present a cropped/labelled region as a new image, then labels added within it could be considered children; perhaps you could say "annotate parts of this dog" to make it clear you shouldn't label parts behind/infront of it in this view.

failing anything like that, just being able to annotate components without grouping would still be useful IMO.

(iv) some suggestions for more component labels:-

• handle (cutlery and many tools or common objects have these, even peices of furniture)
• wheel
• foot
• paw (like foot but specific to animals)
• tail
• tail fin , tail wing (aircraft parts, also fish?)
• hand (mostly 'person' but some animals have them too)
• wing (bird, aircraft, insect)
• headlight (car)
• wing mirror (car)
• exhaust
• intake (on aircraft? sometimes on cars too?)
• arm
• leg (also leg (furniture), leg (animal) leg (person..))
• neck
• lid
• socket (plugs, connectors on the side of a laptop?)
• button/key (one label that could be on/off switch / general purpose contols or keyboard keys?)
• nozzle (e.g. spray can, soap dispenser)