pyOpenSci
Minor updates to reviewer guide
Related to #200, we probably need to add a little language for clarity in the guide for reviewers
- [ ] add language like "please complete your review before posting and please make sure you complete it under two weeks" if we don't have it already
- [ ] possibly add links to previous reviews throughout, as examples to help reviewers?
this is a great idea! i can tag the issue as help wanted in case anyone has time for a small pr!
Hi,
In addition to the reviewer checklist outlined in issue #200, I would like to suggest the inclusion of the following items to enhance the review process:
[ ] Test Documentation: Confirm that tests are well-documented and easy to understand.
[ ] License: Ensure that the package includes an appropriate open-source license file (e.g., MIT).
[ ] Third-Party Dependencies: Verify that all third-party dependencies are clearly listed and comply with the chosen license.
These additions would help reviewers assess the package's compliance with licensing standards and the clarity of its testing framework.
These are good suggestions. I'm going to remove this from the help-wanted list for the time being so we can discuss what to update and where as a reviewer team!!
@akhilkrishnar0 if you are open to starting a discussion in our Slack about this or directing folks here, we can definitely move this forward together. No pressure if you don't have bandwidth!!
Thanks @lwasser I'd be happy to initiate a discussion on Slack and hear the team's thoughts on incorporating these checklist items. I’ll post a summary there and link back to this issue.
I’m opposed to the “comply with third-party licensing” part suggested in https://github.com/pyOpenSci/software-peer-review/issues/201#issuecomment-2822458607. A common case I struggle with is when a plug-in is considered a “single combined program” under the GLP (https://www.gnu.org/licenses/gpl-faq.en.html#GPLPlugins). I do NOT want to have a discussion about that in this issue, but the fact that there are literally thousands of blog posts about that and dozens or hundreds of posts on similar matters on the Pyopensci slack prove that it’s not easy to decide in every case. And I don’t want to get into a situation where a lawyer points to our review and says “PyOpenSci evaluated that and found that our software complies with the third-party license.” That doesn’t mean that we can’t point out obvious problems, but I’m uncomfortable with a check-box that says “reviewed and approved that this complies with third-party licenses”.
@hamogu thank you for raising it here also, I agree that licensing compliance can be a gray area and certainly not something we want reviewers to make legal judgments on. Perhaps instead of phrasing it as "comply with licensing," we could revise it to something like: “Check that third-party dependencies are clearly listed, and note any potentially unclear licensing issues for editors to review.” That way, it stays observational rather than evaluative. Does that feel more comfortable?
That's better. Still, can we just go with "are clearly listed"? It's hard enough to find reviewers and anything that sounds as if we require them to be experts in licensing law (to identify "any potentially unclear issues") won't make it easier. And, given that our editors are also not lawyers, I'm not sure what I, as an editor, should then do on review of an issue raised.
The problem with all the licensing is that even for lawyers it's not easy to give definitive answers since few issues around open-source licenses have actually been tested in court, and even those narrow few cases are concentrated in very few countries (I'm aware of cases in the USA and Germany), while we review software world-wide (admittedly, the countries where e.g. court cases on the GPL have been fought are also the countries that are overrepresented in PyOpenSci's package submissions).
Thanks for this detailed perspective, @hamogu it’s really helpful. I completely agree that keeping things simple and manageable for reviewers is crucial, especially given the complexity and uncertainty surrounding licensing legalities worldwide.
Updating the checklist item to simply “Check that third-party dependencies are clearly listed” sounds like the best way forward.
Hi,
In addition to the reviewer checklist outlined in issue #200, I would like to suggest the inclusion of the following items to enhance the review process:
[ ] Test Documentation: Confirm that tests are well-documented and easy to understand.
[ ] License: Ensure that the package includes an appropriate open-source license file (e.g., MIT).
[ ] Third-Party Dependencies: Verify that all third-party dependencies are clearly listed.
These additions would help reviewers assess the package's compliance with licensing standards and the clarity of its testing framework.