[REVIEW]: strucscan: A lightweight Python-based framework for high-throughput material simulation

[editorialbot]

Submitting author: @thohamm (Thomas Hammerschmidt) Repository: https://github.com/ICAMS/strucscan Branch with paper.md (empty if default branch): Version: 1.0 Editor: @ppxasjsm Reviewers: @mturiansky, @wcwitt Archive: Pending

Status

Status badge code:

HTML: <a href="https://joss.theoj.org/papers/cf152ba42d55db682d1ac29f951bcfe1"><img src="https://joss.theoj.org/papers/cf152ba42d55db682d1ac29f951bcfe1/status.svg"></a>
Markdown: [![status](https://joss.theoj.org/papers/cf152ba42d55db682d1ac29f951bcfe1/status.svg)](https://joss.theoj.org/papers/cf152ba42d55db682d1ac29f951bcfe1)

Reviewers and authors:

Please avoid lengthy details of difficulties in the review thread. Instead, please create a new issue in the target repository and link to those issues (especially acceptance-blockers) by leaving comments in the review thread below. (For completists: if the target issue tracker is also on GitHub, linking the review thread in the issue or vice versa will create corresponding breadcrumb trails in the link target.)

Reviewer instructions & questions

@mturiansky & @wcwitt, your review will be checklist based. Each of you will have a separate checklist that you should update when carrying out your review. First of all you need to run this command in a separate comment to create the checklist:

@editorialbot generate my checklist

The reviewer guidelines are available here: https://joss.readthedocs.io/en/latest/reviewer_guidelines.html. Any questions/concerns please let @ppxasjsm know.

✨ Please start on your review when you are able, and be sure to complete your review in the next six weeks, at the very latest ✨

Checklists

📝 Checklist for @mturiansky

📝 Checklist for @wcwitt

[editorialbot]

Hello humans, I'm @editorialbot, a robot that can help you with some common editorial tasks.

For a list of things I can do to help you, just type:

@editorialbot commands

For example, to regenerate the paper pdf after making changes in the paper's md or bib files, type:

@editorialbot generate pdf

[editorialbot]

Software report:

github.com/AlDanial/cloc v 1.88  T=0.10 s (673.6 files/s, 97655.0 lines/s)
--------------------------------------------------------------------------------
Language                      files          blank        comment           code
--------------------------------------------------------------------------------
Python                           26            907           1978           3374
Markdown                          9            203              0            910
TeX                               1             12              0            123
Jupyter Notebook                  4              0           1763            121
reStructuredText                  6             37             41             76
YAML                              8             20            135             65
Bourne Again Shell               14             25            137             57
make                              1              5              5              9
--------------------------------------------------------------------------------
SUM:                             69           1209           4059           4735
--------------------------------------------------------------------------------


gitinspector failed to run statistical information for the repository

[editorialbot]

Reference check summary (note 'MISSING' DOIs are suggestions that need verification):

OK DOIs

- 10.1016/0927-0256(96)00008-0 is OK
- 10.1103/PhysRevB.59.1758 is OK
- 10.1103/physrevb.54.11169 is OK
- 10.1016/j.commatsci.2021.110731 is OK
- 10.1016/j.commatsci.2018.07.043 is OK
- 10.1016/j.commatsci.2017.07.030 is OK
- 10.1002/cpe.3505 is OK
- 10.1109/CCGRID.2001.923173 is OK
- 10.1007/10968987_3 is OK
- 10.1088/1361-648x/aa680e is OK
- 10.1088/1367-2630/15/11/115016 is OK
- 10.1038/s41597-020-00638-4 is OK
- 10.1016/j.commatsci.2012.10.028 is OK

MISSING DOIs

- None

INVALID DOIs

- None

[editorialbot]

Wordcount for paper.md is 1446

[editorialbot]

:point_right::page_facing_up: Download article proof :page_facing_up: View article proof on GitHub :page_facing_up: :point_left:

[ppxasjsm]

@mturiansky, @wcwitt, Please go ahead and start your review by generating your checklist. Do let me know if you have any problems/issues in the meantime.

[mturiansky]

Review checklist for @mturiansky

Conflict of interest

• [x] I confirm that I have read the JOSS conflict of interest (COI) policy and that: I have no COIs with reviewing this work or that any perceived COIs have been waived by JOSS for the purpose of this review.

Code of Conduct

• [x] I confirm that I read and will adhere to the JOSS code of conduct.

General checks

• [x] Repository: Is the source code for this software available at the https://github.com/ICAMS/strucscan?
• [x] License: Does the repository contain a plain-text LICENSE file with the contents of an OSI approved software license?
• [x] Contribution and authorship: Has the submitting author (@thohamm) made major contributions to the software? Does the full list of paper authors seem appropriate and complete?
• [x] Substantial scholarly effort: Does this submission meet the scope eligibility described in the JOSS guidelines
• [x] Data sharing: If the paper contains original data, data are accessible to the reviewers. If the paper contains no original data, please check this item.
• [x] Reproducibility: If the paper contains original results, results are entirely reproducible by reviewers. If the paper contains no original results, please check this item.
• [x] Human and animal research: If the paper contains original data research on humans subjects or animals, does it comply with JOSS's human participants research policy and/or animal research policy? If the paper contains no such data, please check this item.

Functionality

• [x] Installation: Does installation proceed as outlined in the documentation?
• [x] Functionality: Have the functional claims of the software been confirmed?
• [x] Performance: If there are any performance claims of the software, have they been confirmed? (If there are no claims, please check off this item.)

Documentation

• [x] A statement of need: Do the authors clearly state what problems the software is designed to solve and who the target audience is?
• [x] Installation instructions: Is there a clearly-stated list of dependencies? Ideally these should be handled with an automated package management solution.
• [x] Example usage: Do the authors include examples of how to use the software (ideally to solve real-world analysis problems).
• [x] Functionality documentation: Is the core functionality of the software documented to a satisfactory level (e.g., API method documentation)?
• [x] Automated tests: Are there automated tests or manual steps described so that the functionality of the software can be verified?
• [x] Community guidelines: Are there clear guidelines for third parties wishing to 1) Contribute to the software 2) Report issues or problems with the software 3) Seek support

Software paper

• [x] Summary: Has a clear description of the high-level functionality and purpose of the software for a diverse, non-specialist audience been provided?
• [x] A statement of need: Does the paper have a section titled 'Statement of need' that clearly states what problems the software is designed to solve, who the target audience is, and its relation to other work?
• [x] State of the field: Do the authors describe how this software compares to other commonly-used packages?
• [x] Quality of writing: Is the paper well written (i.e., it does not require editing for structure, language, or writing quality)?
• [x] References: Is the list of references complete, and is everything cited appropriately that should be cited (e.g., papers, datasets, software)? Do references in the text use the proper citation syntax?

[wcwitt]

Review checklist for @wcwitt

Conflict of interest

• [x] I confirm that I have read the JOSS conflict of interest (COI) policy and that: I have no COIs with reviewing this work or that any perceived COIs have been waived by JOSS for the purpose of this review.

Code of Conduct

• [x] I confirm that I read and will adhere to the JOSS code of conduct.

General checks

• [x] Repository: Is the source code for this software available at the https://github.com/ICAMS/strucscan?
• [x] License: Does the repository contain a plain-text LICENSE file with the contents of an OSI approved software license?
• [x] Contribution and authorship: Has the submitting author (@thohamm) made major contributions to the software? Does the full list of paper authors seem appropriate and complete?
• [x] Substantial scholarly effort: Does this submission meet the scope eligibility described in the JOSS guidelines
• [x] Data sharing: If the paper contains original data, data are accessible to the reviewers. If the paper contains no original data, please check this item.
• [x] Reproducibility: If the paper contains original results, results are entirely reproducible by reviewers. If the paper contains no original results, please check this item.
• [x] Human and animal research: If the paper contains original data research on humans subjects or animals, does it comply with JOSS's human participants research policy and/or animal research policy? If the paper contains no such data, please check this item.

Functionality

• [x] Installation: Does installation proceed as outlined in the documentation?
• [x] Functionality: Have the functional claims of the software been confirmed?
• [x] Performance: If there are any performance claims of the software, have they been confirmed? (If there are no claims, please check off this item.)

Documentation

• [x] A statement of need: Do the authors clearly state what problems the software is designed to solve and who the target audience is?
• [x] Installation instructions: Is there a clearly-stated list of dependencies? Ideally these should be handled with an automated package management solution.
• [x] Example usage: Do the authors include examples of how to use the software (ideally to solve real-world analysis problems).
• [x] Functionality documentation: Is the core functionality of the software documented to a satisfactory level (e.g., API method documentation)?
• [x] Automated tests: Are there automated tests or manual steps described so that the functionality of the software can be verified?
• [x] Community guidelines: Are there clear guidelines for third parties wishing to 1) Contribute to the software 2) Report issues or problems with the software 3) Seek support

Software paper

• [x] Summary: Has a clear description of the high-level functionality and purpose of the software for a diverse, non-specialist audience been provided?
• [x] A statement of need: Does the paper have a section titled 'Statement of need' that clearly states what problems the software is designed to solve, who the target audience is, and its relation to other work?
• [x] State of the field: Do the authors describe how this software compares to other commonly-used packages?
• [x] Quality of writing: Is the paper well written (i.e., it does not require editing for structure, language, or writing quality)?
• [x] References: Is the list of references complete, and is everything cited appropriately that should be cited (e.g., papers, datasets, software)? Do references in the text use the proper citation syntax?

[ppxasjsm]

/ooo October 4 until October 13

[thohamm]

issues fixed and closed

[mturiansky]

I opened one last issue. Once this is addressed, I can tick off the functionality checkbox and am okay with publication. The test coverage is a bit low in my opinion (see below), but I think it does cover the basic functionality.

Name                                Stmts   Miss  Cover
-------------------------------------------------------
strucscan/__init__.py                  10      0   100%
strucscan/_version.py                 337    213    37%
strucscan/cli.py                       35     35     0%
strucscan/core/__init__.py              0      0   100%
strucscan/core/collector.py            28      0   100%
strucscan/core/datatree.py             37      4    89%
strucscan/core/jobmaker.py            138     66    52%
strucscan/core/jobmanager.py          252     61    76%
strucscan/core/jobobject.py            50      5    90%
strucscan/core/statusmanager.py        28     11    61%
strucscan/engine/__init__.py            0      0   100%
strucscan/engine/dummy.py             115     70    39%
strucscan/engine/generalengine.py      63     22    65%
strucscan/engine/vasp.py              244    215    12%
strucscan/error/__init__.py             0      0   100%
strucscan/error/errorhandler.py       107     93    13%
strucscan/error/errormanager.py        27     24    11%
strucscan/properties/__init__.py        0      0   100%
strucscan/properties/bulk.py           11      8    27%
strucscan/properties/eos.py            41     25    39%
strucscan/resources/__init__.py         0      0   100%
strucscan/resources/atomicdata.py       2      0   100%
strucscan/resources/inputyaml.py       22      0   100%
strucscan/resources/properties.py      32      5    84%
strucscan/scheduler.py                168    119    29%
strucscan/utils.py                    236    108    54%
-------------------------------------------------------
TOTAL                                1983   1084    45%

[thohamm]

Thank you for the careful testing of our code and your feedback! We fixed and closed the issue, see comment in github repo.

[wcwitt]

Sorry to be a bit slow here - will finish ASAP

[thohamm]

Thank you also for your comment on the test coverage. The reason is that several parts of the code rely on operating with a scheduler system and are therefore a bit hard to test in a standalone run.

[mturiansky]

I'm all done with my review now, @ppxasjsm . All my concerns were addressed, and I'm okay with publication.

[thohamm]

@mturiansky Thank you for reviewing our work and for your constructive comments.

[ppxasjsm]

Thank you @mturiansky for your review and your feedback. @wcwitt can you give me an estimate of when you think you'll be able to finish the reveiw by? Thanks.

[wcwitt]

I'll finish this week.

[ppxasjsm]

Brilliant! Thank you.

[wcwitt]

I've now managed to install and run the Dummy examples, both from the Jupyter notebook and the command line. I don't have a VASP license, so I can't verify those parts. Overall, I think the submission is of good quality. There are two issues (linked above) for the authors to respond to.

[thohamm]

@wcwitt Thank you for your review and your constructive feedback. We responded to the issues with comments and pushed according changes to the git repo.

[wcwitt]

I've reviewed the changes - happy for publication at this stage.

[ppxasjsm]

Thank you! @thohamm can you go through the PDF of the paper version and do a check to make sure you are happy with the current state? I'll also go through it one last time, but won't likely get a chance to do so until next week I am afraid.

[thohamm]

@editorialbot generate pdf

[editorialbot]

:point_right::page_facing_up: Download article proof :page_facing_up: View article proof on GitHub :page_facing_up: :point_left:

[pietki8q]

@editorialbot generate pdf

[editorialbot]

:point_right::page_facing_up: Download article proof :page_facing_up: View article proof on GitHub :page_facing_up: :point_left:

[pietki8q]

@editorialbot generate pdf

[editorialbot]

:point_right::page_facing_up: Download article proof :page_facing_up: View article proof on GitHub :page_facing_up: :point_left:

[thohamm]

@editorialbot generate pdf