openjournals
[REVIEW]: Can you predict the future? A tutorial for the National Ecological Observatory Network Ecological Forecasting Challenge
Submitting author: @OlssonF (Freya Olsson) Repository: https://github.com/OlssonF/NEON-forecast-challenge-workshop Branch with paper.md (empty if default branch): main Version: v1.1.0 Editor: @acocac Reviewers: @dlebauer, @skmorgane Archive: Pending Paper kind: learning module
Status
Status badge code:
HTML: <a href="https://jose.theoj.org/papers/37e9e48c49abf89a619b787d106d79bd"><img src="https://jose.theoj.org/papers/37e9e48c49abf89a619b787d106d79bd/status.svg"></a>
Markdown: [](https://jose.theoj.org/papers/37e9e48c49abf89a619b787d106d79bd)
Reviewers and authors:
Please avoid lengthy details of difficulties in the review thread. Instead, please create a new issue in the target repository and link to those issues (especially acceptance-blockers) by leaving comments in the review thread below. (For completists: if the target issue tracker is also on GitHub, linking the review thread in the issue or vice versa will create corresponding breadcrumb trails in the link target.)
Reviewer instructions & questions
@dlebauer & @acocac, your review will be checklist based. Each of you will have a separate checklist that you should update when carrying out your review. First of all you need to run this command in a separate comment to create the checklist:
@editorialbot generate my checklist
The reviewer guidelines are available here: https://openjournals.readthedocs.io/en/jose/reviewer_guidelines.html. Any questions/concerns please let @acocac know.
✨ Please start on your review when you are able, and be sure to complete your review in the next six weeks, at the very latest ✨
Checklists
@dlebauer, please create your checklist typing: @editorialbot generate my checklist
@acocac, please create your checklist typing: @editorialbot generate my checklist
Hello humans, I'm @editorialbot, a robot that can help you with some common editorial tasks.
For a list of things I can do to help you, just type:
@editorialbot commands
For example, to regenerate the paper pdf after making changes in the paper's md or bib files, type:
@editorialbot generate pdf
Reference check summary (note 'MISSING' DOIs are suggestions that need verification):
✅ OK DOIs
- 10.5281/zenodo.8316965 is OK
- 10.1002/fee.2616 is OK
- 10.1002/ecs2.2567 is OK
- 10.21105/joss.01686 is OK
- 10.1073/pnas.1710231115 is OK
- 10.1111/2041-210X.13955 is OK
- 10.1515/9781400885459 is OK
- 10.21105/jose.00198 is OK
🟡 SKIP DOIs
- None
❌ MISSING DOIs
- None
❌ INVALID DOIs
- None
Software report:
github.com/AlDanial/cloc v 1.90 T=0.02 s (755.2 files/s, 241591.4 lines/s)
-------------------------------------------------------------------------------
Language files blank comment code
-------------------------------------------------------------------------------
Markdown 5 464 0 1822
Rmd 3 366 648 511
R 3 53 44 114
TeX 1 9 0 105
YAML 1 1 4 18
-------------------------------------------------------------------------------
SUM: 13 893 696 2570
-------------------------------------------------------------------------------
Commit count by author:
151 LAPTOP-FO87M896reya
28 Freya Olsson
4 rqthomas
3 github-actions
2 Cayelan Carey
1 Carl Boettiger
1 Mary Lofton
1 Quinn Thomas
1 QuinnThomas
Paper file info:
📄 Wordcount for paper.md is 1662
✅ The paper includes a Statement of need section
:point_right::page_facing_up: Download article proof :page_facing_up: View article proof on GitHub :page_facing_up: :point_left:
Hi @OlssonF fyi - I got a late reply from one of your suggested reviewers, Morgan Ernest, who has confirmed availability within a time period of 3 weeks. I hope this is fine for you. I'll remove myself from the reviewer list.
👋 @dlebauer @skmorgane we will conduct the review in this issue.
Please read through the above information and let me know if you have any questions about the review process.
Thank you 🙏
Review checklist for @dlebauer
Conflict of interest
- [x] As the reviewer I confirm that I have read the JOSE conflict of interest policy and that there are no conflicts of interest for me to review this work.
Code of Conduct
- [x] I confirm that I read and will adhere to the JOSE code of conduct.
General checks
- [x] Repository: Is the source for this learning module available at the https://github.com/OlssonF/NEON-forecast-challenge-workshop?
- [x] License: Does the repository contain a plain-text LICENSE file with the contents of a standard license? (OSI-approved for code, Creative Commons for content)
- [x] Version: Does the release version given match the repository release?
- [x] Authorship: Has the submitting author (@OlssonF) made visible contributions to the module? Does the full list of authors seem appropriate and complete?
Documentation
- [x] A statement of need: Do the authors clearly state the need for this module and who the target audience is?
- [x] Installation instructions: Is there a clearly stated list of dependencies?
- [x] Usage: Does the documentation explain how someone would adopt the module, and include examples of how to use it?
- [x] Community guidelines: Are there clear guidelines for third parties wishing to 1) Contribute to the module 2) Report issues or problems with the module 3) Seek support
Pedagogy / Instructional design (Work-in-progress: reviewers, please comment!)
- [x] Learning objectives: Does the module make the learning objectives plainly clear? (We don't require explicitly written learning objectives; only that they be evident from content and design.)
- [x] Content scope and length: Is the content substantial for learning a given topic? Is the length of the module appropriate?
- [x] Pedagogy: Does the module seem easy to follow? Does it observe guidance on cognitive load? (working memory limits of 7 +/- 2 chunks of information)
- [x] Content quality: Is the writing of good quality, concise, engaging? Are the code components well crafted? Does the module seem complete?
- [x] Instructional design: Is the instructional design deliberate and apparent? For example, exploit worked-example effects; effective multi-media use; low extraneous cognitive load.
JOSE paper
- [x] Authors: Does the
paper.mdfile include a list of authors with their affiliations? - [x] A statement of need: Does the paper clearly state the need for this module and who the target audience is?
- [x] Description: Does the paper describe the learning materials and sequence?
- [x] Does it describe how it has been used in the classroom or other settings, and how someone might adopt it?
- [x] Could someone else teach with this module, given the right expertise?
- [x] Does the paper tell the "story" of how the authors came to develop it, or what their expertise is?
- [x] References: Do all archival references that should have a DOI list one (e.g., papers, datasets, software)?
@acocac Could you please help me understand the version requirement "Does the release version given match the repository release?" - the latest release in the repository is v1.2.0, this issue lists it as v1.1.0. It isn't clear to me which version the proof is from, though I suspect any changes will at least bump the version.
--> I think it would be helpful to clarify this part of the checklist and reviewer instructions
Hello @acocac, here are the things you can ask me to do:
# List all available commands
@editorialbot commands
# Add to this issue's reviewers list
@editorialbot add @username as reviewer
# Remove from this issue's reviewers list
@editorialbot remove @username from reviewers
# Get a list of all editors's GitHub handles
@editorialbot list editors
# Assign a user as the editor of this submission
@editorialbot assign @username as editor
# Remove the editor assigned to this submission
@editorialbot remove editor
# Remind an author, a reviewer or the editor to return to a review after a
# certain period of time (supported units days and weeks)
@editorialbot remind @reviewer in 2 weeks
# Adds a checklist for the reviewer using this command
@editorialbot generate my checklist
# Set a value for version
@editorialbot set v1.0.0 as version
# Set a value for branch
@editorialbot set jose-paper as branch
# Set a value for repository
@editorialbot set https://github.com/organization/repo as repository
# Set a value for the archive DOI
@editorialbot set 10.5281/zenodo.6861996 as archive
# Mention the EiCs for the correct track
@editorialbot ping track-eic
# Run checks and provide information on the repository and the paper file
@editorialbot check repository
# Check the references of the paper for missing DOIs
@editorialbot check references
# Generates the pdf paper
@editorialbot generate pdf
# Recommends the submission for acceptance
@editorialbot recommend-accept
# Generates a LaTeX preprint file
@editorialbot generate preprint
# Flag submission with questionable scope
@editorialbot query scope
# Get a link to the complete list of reviewers
@editorialbot list reviewers
# Creates a post-review checklist with editor and authors tasks
@editorialbot create post-review checklist
# Open the review issue
@editorialbot start review
@acocac Could you please help me understand the version requirement "Does the release version given match the repository release?" - the latest release in the repository is v1.2.0, this issue lists it as v1.1.0. It isn't clear to me which version the proof is from, though I suspect any changes will at least bump the version.
--> I think it would be helpful to clarify this part of the checklist and reviewer instructions
@dlebauer - I've fixed the issue. The revision should be on the latest release.
Review checklist for @skmorgane
Conflict of interest
- [x] As the reviewer I confirm that I have read the JOSE conflict of interest policy and that there are no conflicts of interest for me to review this work.
Code of Conduct
- [x] I confirm that I read and will adhere to the JOSE code of conduct.
General checks
- [x] Repository: Is the source for this learning module available at the https://github.com/OlssonF/NEON-forecast-challenge-workshop?
- [x] License: Does the repository contain a plain-text LICENSE file with the contents of a standard license? (OSI-approved for code, Creative Commons for content)
- [x] Version: Does the release version given match the repository release?
- [x] Authorship: Has the submitting author (@OlssonF) made visible contributions to the module? Does the full list of authors seem appropriate and complete?
Documentation
- [x] A statement of need: Do the authors clearly state the need for this module and who the target audience is?
- [x] Installation instructions: Is there a clearly stated list of dependencies?
- [x] Usage: Does the documentation explain how someone would adopt the module, and include examples of how to use it?
- [ ] Community guidelines: Are there clear guidelines for third parties wishing to 1) Contribute to the module 2) Report issues or problems with the module 3) Seek support
Pedagogy / Instructional design (Work-in-progress: reviewers, please comment!)
- [x] Learning objectives: Does the module make the learning objectives plainly clear? (We don't require explicitly written learning objectives; only that they be evident from content and design.)
- [x] Content scope and length: Is the content substantial for learning a given topic? Is the length of the module appropriate?
- [x] Pedagogy: Does the module seem easy to follow? Does it observe guidance on cognitive load? (working memory limits of 7 +/- 2 chunks of information)
- [x] Content quality: Is the writing of good quality, concise, engaging? Are the code components well crafted? Does the module seem complete?
- [x] Instructional design: Is the instructional design deliberate and apparent? For example, exploit worked-example effects; effective multi-media use; low extraneous cognitive load.
JOSE paper
- [x] Authors: Does the
paper.mdfile include a list of authors with their affiliations? - [x] A statement of need: Does the paper clearly state the need for this module and who the target audience is?
- [x] Description: Does the paper describe the learning materials and sequence?
- [x] Does it describe how it has been used in the classroom or other settings, and how someone might adopt it?
- [x] Could someone else teach with this module, given the right expertise?
- [x] Does the paper tell the "story" of how the authors came to develop it, or what their expertise is?
- [x] References: Do all archival references that should have a DOI list one (e.g., papers, datasets, software)?
@acocac I started my review by making sure I didn't have conflicts with the full author list but realized that two of the authors I have an association with that I should disclose: Carey and Thomas. We are coauthors on a large author line paper that was just accepted. I think this one falls under "co-authors but not really collaborating" in the guidelines and is probably not a concern. However, we are also collaborating on a proposal, again a large multi-investigator/multi-institutional effort, but we do interact a little more directly there. I think I can review this impartially, but if the association violates JOSE rules, I have two people I can recommend as reviewers who I know are not interacting with anyone on the author line. Just let me know how you would like to proceed
@acocac I started my review by making sure I didn't have conflicts with the full author list but realized that two of the authors I have an association with that I should disclose: Carey and Thomas. We are coauthors on a large author line paper that was just accepted. I think this one falls under "co-authors but not really collaborating" in the guidelines and is probably not a concern. However, we are also collaborating on a proposal, again a large multi-investigator/multi-institutional effort, but we do interact a little more directly there. I think I can review this impartially, but if the association violates JOSE rules, I have two people I can recommend as reviewers who I know are not interacting with anyone on the author line. Just let me know how you would like to proceed
@labarba can you assist on this? Thank you
@skmorgane – Thank you for raising the potential conflict with some authors. Given that JOSE reviews are open, done in public, meant to be constructive, I am inclined to waive the COI in this instance. Our review process here is not a gatekeeping exercise, and authors recommend publication once all items in a review checklist are passed. Open peer review for the win!
Please go ahead and complete your review—and thank you kindly for your contributions 🙏
I have finished my review of the NEON-forecast-challenge-workshop by Olsson et al.
Overall I think this is a really useful contribution to the ecological forecasting teaching materials. Students in my class have often requested that we add something like this to our course but the energy it would take to design and implement has prevented us from doing that. I reviewed the materials with that in mind and found them to be well organized, easy to understand, and well focused on the essential skills students would need to be introduced to. I can easily see implementing these materials either as is or with minor modifications to reflect the focus of my class.
The only box I on the review sheet that I could not tick was the community engagement statement.
I found only minor issues with the materials and left issues in the source repo for the author's consideration.
Thank you all for your time to review these materials! @acocac what are the steps that we should take for making revisions following the Reviewer comments? They having submitted pull requests and Github issues that we can address, but is there additional responses that should be completed? Thanks
@skmorgane thanks for confirming about the status of the review!
@dlebauer could you confirm if your checklist is complete? Do you have any general feedback about the submission?
Thank you all for your time to review these materials! @acocac what are the steps that we should take for making revisions following the Reviewer comments? They having submitted pull requests and Github issues that we can address, but is there additional responses that should be completed? Thanks
Please address the reviewers comments, issues and PRs. Then, I'll double check if they're support the publication in JOSE. Thank you.
Thanks @acocac. We are also wanting to change the ownership of the repository to ensure long-term sustainability and use of the materials. We would like to move the materials from my personal organisation (@OlssonF) to the eco4cast organisation (https://github.com/eco4cast). Does it matter if this done before or after publication? I believe the transfer will not impact previous users interactions or forks etc. Thanks
The repository is linked from the paper landing page and cover, so best to transfer before publication.
@OlssonF please transfer to the organisation as soon as you can, and implement the changes suggested by the reviewers. Please notify here when everything is ready.