Dedicated Reviewing Interface

[seveibar]

Let's begin as a discussion:

What do people want in an interface that allows a reviewer to enter a collaborative session and review the work of labelers? What would be most useful?

[mh-mazen]

i already started working on a similar interface prototype , its a modified version of the annotation tool it self , supports localization and the first features request was to add an accept all button

[seveibar]

very cool! This looks like a modification of the <RegionLabel /> component! I don't know if we should start with reviewing individual labels or if we should have a built-in workflow system

[terminalh2t3]

I used to try some other paid data annotation tools and they usually offered some cool quality control features. For example, Labelbox offered Consensus checking, Gold Standard, Label feedback and then they can tracking and monitoring the labeler in real-time (CVAT used Kibana).