Added support of skeletons

[bsekachev]

TODO for client side:

Enhancements:

• [x] Rotation for skeletons
• [x] Undo/redo when updating a skeleton or its parts
• [x] Enable z order
• [x] Add ability to hide one point (shortcut)
• [x] Occluded/pinned/outside for a skeleton and its parts
• [x] Enable attributes
• [x] Allow editing skeletons in constructor
• [x] Merge/split/group
• [x] Add setup to hide/display text labels for skeleton parts
• [x] Add setup to points size
• [x] Support filtration

Known bugs:

• [x] List of "internal" labels visible outside
• [x] When moving polygon parts, wrapping box is not updated properly
• [x] When moving polygon parts out of wrapping box, other parts can be shifted
• [x] Exception: The field "tag count" is required for "Send task info" log
• [x] Shape activation margin is 0, but drawn margin is not 0 for wrapping box
• [x] Rotation property is not applied during pasting a skeleton
• [x] Cannot save tracks on the server (wrong output from client side)
• [x] Colors for points are not valid when "Color by" set to "label"
• [x] Upload skeleton button does not work properly in skeleton configurator

Future list:

• [ ] Support colorizing skeleton edges
• [ ] Integrate with attribute annotation mode
• [ ] Support rectangles inside skeleton
• [ ] Support a DL model to semi-automatic skeleton annotations (one use-case)
• [ ] Support editing skeletons

If you need more, please submit an issue.

TODO for server side:

• [x] Interpolation
• [x] Extend CVAT for video (import)
• [x] Backups

[nmanovic]

Hi,

There are several resources. I will add all of them into README file ASAP.

• Basic information + how to build and run: https://github.com/opencv/cvat/blob/master/README.md
• User's guide (github cannot render images at the moment, it will be fixed): https://github.com/opencv/cvat/blob/master/cvat/apps/documentation/user_guide.md
• XML format for annotations: https://github.com/opencv/cvat/blob/master/cvat/apps/documentation/xml_format.md
• Computer Vision Annotation Tool (CVAT): interpolation mode screencast: https://www.youtube.com/watch?v=sc5X5hvxNfA
• Computer Vision Annotation Tool (CVAT): attribute mode screencast: https://www.youtube.com/watch?v=5yXaG0V7X0Q
• Computer Vision Annotation Tool (CVAT): annotation mode screencast: https://www.youtube.com/watch?v=uSqaQENdyJE

[headdab]

Thanks. That helped a lot. My first issue was that after I created a task, I didn't realize that all the controls were off the screen - I had to scroll or enlarge my browser window to find them.

A few comments:

• none of the figure work in the user's guide (but i'm guessing you're working on this).

• I couldn't get the docker-compose.override.yml to work properly. I had to add it explicitly to the docker up command.

• docker-compose.yml and docker-compose.override.yml use different port mappings, i.e. 8080:8008 and 80:8080, resp. As such, at least with the explicit inclusion of the override file, both mappings are used.

• i started up a gpu/tf enabled version, but I don't see anything related to tensorflow in the interface and there isn't any documentation about how to use this feature. Seems like a good idea though.

• I don't see anything on how to use the shared mode. When I select it, it makes me choose files, but points to //icv-cifs/icv_projects/cvat/data. One time, when I hit back from a browser (from a session) window it seemed to allow me to add local files from the incomplete task create window.

[bsekachev]

Good day,

• "I didn't realize that all the controls were off the screen". Could you provide any screenshots please?
• "none of the figure work in the user's guide". Yes, we are working on it.
• "i started up a gpu/tf enabled version, but I don't see anything related to tensorflow in the interface and there isn't any documentation about how to use this feature. Seems like a good idea though.". If you did all right, button "Run TF Annotation" should appears in the dashboard for each task. It will run auto annotation for supported labels (if task contain any of such labels). We will include an information about it to documentation.
• "I don't see anything on how to use the shared mode. When I select it, it makes me choose files, but points to //icv-cifs/icv_projects/cvat/data.". This path will be removed. Currently, if you want use share source, you need mount it to /home/django/share in the docker container.

[headdab]

I attached to screenshots - partial is what I saw originally, fullscreen after I enlarged the window. You might be making your layout decisions based on a landscape layout assumption.

I found the tf button later, but got a label error. It makes more sense now. Can you give me an example of the labels string I should use, say, if I want to detect person and sports_ball? I tried:

tag @select=class:person,sports_ball

but that's not right. The schema is:

label_name input_type=attribute_name:attribute_value1,attribute_value2

When you mention labels, you're referring to label_name in the schema? Can I include more than 2 of those in the label string when configuring the session?

Re: shared mode, I'll try your suggestion.

Thanks for the quick feedback.

[headdab]

So I figured out the tf thing. I just used "person sports_ball" and that worked.

[bsekachev]

Thanks a lot, Layout looks strange, we will be working on it. Yes, label is label_name. If you want detect persons and sport balls, you need type "person sports_ball" only. Also you may assign some attributes for labels. Attribute examples: age or gender (for person), type or model (for car) etc. I will give some examples of valid strings below:

# number, select, radio, checkbox, text - is input types
# You also need specify each attribute type (mutable [~] or immutable [@]). Mutable attributes may be changed frame by frame (in the case of tracked objects)
# For number input type, attribute values are min, max and step.
# For radio and select - are possible values
# For checkbox and text - are default attribute value (true, false) for checkbox and ("any text") for text
person @number=age:1,100,1 @select=gender:man,woman @radio=hair:long,short,curly,straight
car ~checkbox=parked:true
face @text=description:"default description" 

Of course, you may create any number of labels and attributes for task:

person @number=age:1,100,1 @select=gender:man,woman @radio=hair:long,short,curly,straight car ~checkbox=parked:true face @text=description:"default description" 

Also you may see help about it if you hover cursor to labels string. Or you can see it in HTML Good luck

[bsekachev]

In order to see the rendered documentation, you need press F1 key in the dashboard (task list).

[headdab]

I tried the shared thing, but may have found a bug. I copied a small clip into the top level host share directory and can see it when I try to select files from the create task dialog, but if I select it it also selects the enclosing directory, which it also treats as a file, which I think is the reason for the error shown on the screenshot. For this screenshot, I clicked on select files after I got the error, so you could see that screen.

The test of the error is:

Could not create the task. ffmpy.FFRuntimeError: ffmpeg -i /home/django/data/5/.upload/ -start_number 0 -b:v 10000k -vsync 0 -an -y -q:v 16 /tmp/cvat-0qn3bhs9.data/%d.jpg exited with status 1 STDOUT: STDERR:

[bsekachev]

Yes, it known bug. Actually it already was fixed, but currently was not released. As temporary solution, I can suggest you to put any other file (empty text file for example) together with video. In this case video only will be selected.

[nmanovic]

@headdab we added information about available documentation + screenshot, screencasts. Also fixed a couple of other issues. If you have another issues please open new issues. In any case we will try to address other issues which you found in next release. Don't hesitate to send us MRs. I will close the thread.

[headdab]

Great. Thanks!