Validation assigned to other reviewer

[esyriani]

Currently, when a validation step is required in a screening phase, we can assign the validation of a paper to a user who has already screened that paper. It is preferable to have it assigned to another user (with validator a role) when possible. There should be an option to allow assigning papers to validate to any validator or to a validator who has not already screened this paper in that phase.

Here is a workaround I currently use:

Manual validation outside of ReLiS

The validation in ReLiS cannot be parametrized by specific exclusion criteria. It also cannot guarantee that a validator is assigned a paper he already screened. Therefore, we define a process to perform the validation manually outside of ReLiS.

1. Find the reviewers who screened papers. We need to map the `user_id` with the `user_name` of each reviewer of the project. The `where` clause is optional.
   

select user_id, user_name from relis_new_2_admin.users where user_id IN (3,488,578,531)

2. Find the exclusion criteria. We need to map the id of each exclusion criteria to their name
   

3. Find all the papers that were screened for a specific exclusion criteria. The exclusion_criteria IN () should enumerate the ids of the exclusion criteria. The screening_time >= can be used if you want to validate papers excluded after a certain date, for example after the calibration phase.

SELECT paper.id FROM screening_paper INNER JOIN ref_exclusioncrieria ON ref_exclusioncrieria.ref_id = screening_paper.exclusion_criteria INNER JOIN paper ON paper.id = screening_paper.paper_id WHERE exclusion_criteria IN (36,37,38) AND ref_active=1 AND screening_time >= '2020-06-20'

You obtain a list of ids. Using a text editor, convert the result into a Python list like so: [1,2,3]

4. Retrieve a sample to validate. We get a random sample of 20% of the excluded papers. In python:

import random papers = [PASTE_THE_LIST_OF_IDS] sample = tuple(random.sample(papers, round(len(papers)*.2))) print(sample) 5. Get all the screening information of the sample to validate. The paper.id IN () should enumerate the output of step 4 (the content of sample).

SELECT paper.id, paper.bibtexKey, paper.title, paper.abstract, REPLACE(REPLACE(paper.preview, '<p>',''), '</p>','') AS preview, paper.doi, screening_time, ref_value AS exclusion, CASE WHEN user_id = 3 THEN 'Alice' WHEN user_id = 488 THEN 'Bob' WHEN user_id = 531 THEN 'Charlie' WHEN user_id = 578 THEN 'Dalia' END AS user_name FROM screening_paper INNER JOIN ref_exclusioncrieria ON ref_exclusioncrieria.ref_id = screening_paper.exclusion_criteria INNER JOIN paper ON paper.id = screening_paper.paper_id WHERE exclusion_criteria IN (36,37,38) AND ref_active=1 AND screening_time >= '2020-06-20' AND paper.id IN (12993, 13587) 6. Export to Excel Paste the result of the query in Excel. Ensure there are no duplicates in the column id. You can add a column to activate all doi hyperlinks using =HYPERLINK(F2).