argilla icon indicating copy to clipboard operation
argilla copied to clipboard

[FEATURE] Allow all annotators in workspace to see all the submitted records

Open cceyda opened this issue 5 months ago • 1 comments

Is your feature request related to a problem? Please describe.

Let's say I have 2 annotators A & B they are both part of an annotators workspace. And I'm the admin/owner. I create a dataset in the annotators workspace, I set rg.TaskDistribution(min_submitted=1) as I only need it labeled once. annotator A gets to work early and submits 1000records, then annotator B starts working... but is unsure how to label certain things so when asked I say; "just check what annotator A has done and follow that rule" (can use similarity search filters etc to find relevant already submitted records)

But in the current UI annotator B cannot see submitted records of annotator A. It used to be all the users of the workspace could see all the records. and I totally understand the new flow divvies up pending records and every annotator gets their own view of their pending/drafs/submits

but this makes any scenario like I described difficult. It also makes it difficult to have a double checker that checks over all the annotations from time to time (Honestly this is where I used search the most! To check label coherence across annotators). Same for the scenarios where annotator A & B may no longer be working and an annotator C has to continue (Despite having clear annotation guidelines it is always useful to be able to refer to already labeled relevant data)

I also understand this cross-annotator visibility may not be wanted in certain cases (like where annotator cheating is a concern)

Describe the solution you'd like Have an option/setting where it makes it possible for all users to see submitted records of all other users in the workspace. (this can be just view only, they don't need to see other's drafts, just submitted/discarded, aka finished, records)

At the very least admin/owner should be able to see all the submitted records by all annotators. Right now even though I'm admin I can see nothing other users submitted

Describe alternatives you've considered Give every user owner status...? Or force people to just use a single user(defeats the purpose of TaskDistribution)

cceyda avatar Sep 19 '24 10:09 cceyda