joss-reviews icon indicating copy to clipboard operation
joss-reviews copied to clipboard

[REVIEW]: PyMPDATA v1: Numba-accelerated Pythonic implementation of MPDATA with examples in Python, Julia and Matlab

Open whedon opened this issue 3 years ago • 88 comments

Submitting author: @slayoo (Sylwester Arabas) Repository: https://github.com/atmos-cloud-sim-uj/PyMPDATA Branch with paper.md (empty if default branch): Version: v1 Editor: @arfon Reviewers: @Chiil, @wdeconinck Archive: Pending

:warning: JOSS reduced service mode :warning:

Due to the challenges of the COVID-19 pandemic, JOSS is currently operating in a "reduced service mode". You can read more about what that means in our blog post.

Status

status

Status badge code:

HTML: <a href="https://joss.theoj.org/papers/10e7361e43785dbb1b3d659c5b01757a"><img src="https://joss.theoj.org/papers/10e7361e43785dbb1b3d659c5b01757a/status.svg"></a>
Markdown: [![status](https://joss.theoj.org/papers/10e7361e43785dbb1b3d659c5b01757a/status.svg)](https://joss.theoj.org/papers/10e7361e43785dbb1b3d659c5b01757a)

Reviewers and authors:

Please avoid lengthy details of difficulties in the review thread. Instead, please create a new issue in the target repository and link to those issues (especially acceptance-blockers) by leaving comments in the review thread below. (For completists: if the target issue tracker is also on GitHub, linking the review thread in the issue or vice versa will create corresponding breadcrumb trails in the link target.)

Reviewer instructions & questions

@dmikushin & @olekravchenko, please carry out your review in this issue by updating the checklist below. If you cannot edit the checklist please:

  1. Make sure you're logged in to your GitHub account
  2. Be sure to accept the invite at this URL: https://github.com/openjournals/joss-reviews/invitations

The reviewer guidelines are available here: https://joss.readthedocs.io/en/latest/reviewer_guidelines.html. Any questions/concerns please let @mjsottile know.

Please start on your review when you are able, and be sure to complete your review in the next six weeks, at the very latest

whedon avatar Nov 07 '21 07:11 whedon

Hello human, I'm @whedon, a robot that can help you with some common editorial tasks. @dmikushin, @olekravchenko it looks like you're currently assigned to review this paper :tada:.

:warning: JOSS reduced service mode :warning:

Due to the challenges of the COVID-19 pandemic, JOSS is currently operating in a "reduced service mode". You can read more about what that means in our blog post.

:star: Important :star:

If you haven't already, you should seriously consider unsubscribing from GitHub notifications for this (https://github.com/openjournals/joss-reviews) repository. As a reviewer, you're probably currently watching this repository which means for GitHub's default behaviour you will receive notifications (emails) for all reviews 😿

To fix this do the following two things:

  1. Set yourself as 'Not watching' https://github.com/openjournals/joss-reviews:

watching

  1. You may also like to change your default settings for this watching repositories in your GitHub profile here: https://github.com/settings/notifications

notifications

For a list of things I can do to help you, just type:

@whedon commands

For example, to regenerate the paper pdf after making changes in the paper's md or bib files, type:

@whedon generate pdf

whedon avatar Nov 07 '21 07:11 whedon

PDF failed to compile for issue #3896 with the following error:

 Can't find any papers to compile :-(

whedon avatar Nov 07 '21 07:11 whedon

Software report (experimental):

github.com/AlDanial/cloc v 1.88  T=0.11 s (659.4 files/s, 49581.6 lines/s)
-------------------------------------------------------------------------------
Language                     files          blank        comment           code
-------------------------------------------------------------------------------
Python                          60            586            375           3462
Markdown                         1             93              0            527
YAML                            10             33              6            257
-------------------------------------------------------------------------------
SUM:                            71            712            381           4246
-------------------------------------------------------------------------------


Statistical information for the repository '8066b4e37f38bca61302e6b0' was
gathered on 2021/11/07.
The following historical commit information, by author, was found:

Author                     Commits    Insertions      Deletions    % of changes
Maciej Manna                     9           196            263            1.06
Michael                         93          1687           1332            6.97
Michaeldz36                     38          3704           1752           12.60
Sylwester Arabas               334         14308           9780           55.61
kruci-no                         2            22              4            0.06
piotrbartman                     1            37           1207            2.87
prbartman                       42          3780           5245           20.83

Below are the number of rows from each author that have survived and are still
intact in the current revision:

Author                     Rows      Stability          Age       % in comments
Michael                     109            6.5         17.4               10.09
Sylwester Arabas           4095           28.6          5.4                8.03
kruci-no                      4           18.2         14.6                0.00
piotrbartman                215          581.1         17.5                2.33

whedon avatar Nov 07 '21 07:11 whedon

@whedon generate pdf from branch JOSS

mjsottile avatar Nov 07 '21 07:11 mjsottile

Attempting PDF compilation from custom branch JOSS. Reticulating splines etc...

whedon avatar Nov 07 '21 07:11 whedon

:point_right::page_facing_up: Download article proof :page_facing_up: View article proof on GitHub :page_facing_up: :point_left:

whedon avatar Nov 07 '21 07:11 whedon

@whedon check references

mjsottile avatar Nov 07 '21 07:11 mjsottile

@whedon check references from branch JOSS

mjsottile avatar Nov 07 '21 22:11 mjsottile

Attempting to check references... from custom branch JOSS

whedon avatar Nov 07 '21 22:11 whedon

Reference check summary (note 'MISSING' DOIs are suggestions that need verification):

OK DOIs

- 10.1175/1520-0493(1983)111<0479:ASPDAS>2.0.CO;2 is OK
- 10.1016/0021-9991(84)90121-9 is OK
- 10.1016/0021-9991(86)90270-6  is OK
- 10.1016/0021-9991(90)90105-A is OK
- 10.1175/1520-0493(1993)121<1847:OFITDF>2.0.CO;2 is OK
- 10.1201/9780203711194 is OK
- 10.1006/jcph.1998.5901 is OK
- 10.1137/S106482759324700X is OK
- 10.1016/j.jcp.2004.12.021 is OK
- 10.1002/fld.1070 is OK
- doi:10.1002/fld.1071 is OK
- 10.1002/qj.1913 is OK
- 10.3233/SPR-140379 is OK
- 10.1016/j.jcp.2015.02.003 is OK
- 10.5194/gmd-8-1005-2015 is OK
- 10.1007/978-3-319-93864-6_5 is OK
- 10.5194/gmd-12-651-2019 is OK
- 10.1016/j.cam.2019.05.023 is OK
- 10.5194/gmd-2020-404 is OK

MISSING DOIs

- None

INVALID DOIs

- 10.1175/1520-0450(1968)007%3C0160:AOFDMA%3E2.0.CO;2 is INVALID
- 10.1175/1520-0493(1989)117%3C0102:TDSLTW%3E2.0.CO;2 is INVALID

whedon avatar Nov 07 '21 22:11 whedon

it's puzzling as the above "INVALID DOIs" seem correct, at leat both of the below URLs redirect correctly to the journal site:

  • https://doi.org/10.1175/1520-0450(1968)007%3C0160:AOFDMA%3E2.0.CO;2
  • https://doi.org/10.1175/1520-0493(1989)117%3C0102:TDSLTW%3E2.0.CO;2

slayoo avatar Nov 07 '21 23:11 slayoo

Please make sure the DOIs are in doi entries in the bib file rather than url entries, and that they don't include the https://doi.org part

danielskatz avatar Nov 07 '21 23:11 danielskatz

And perhaps change the characters that are being encoded for example:

10.1175/1520-0493(1989)117<0102:TDSLTW>2.0.CO;2

danielskatz avatar Nov 07 '21 23:11 danielskatz

@whedon check references from branch JOSS

slayoo avatar Nov 08 '21 05:11 slayoo

Attempting to check references... from custom branch JOSS

whedon avatar Nov 08 '21 05:11 whedon

Reference check summary (note 'MISSING' DOIs are suggestions that need verification):

OK DOIs

- 10.1175/1520-0450(1968)007<0160:AOFDMA>2.0.CO;2 is OK
- 10.1175/1520-0493(1983)111<0479:ASPDAS>2.0.CO;2 is OK
- 10.1016/0021-9991(84)90121-9 is OK
- 10.1016/0021-9991(86)90270-6  is OK
- 10.1175/1520-0493(1989)117<0102:TDSLTW>2.0.CO;2 is OK
- 10.1016/0021-9991(90)90105-A is OK
- 10.1175/1520-0493(1993)121<1847:OFITDF>2.0.CO;2 is OK
- 10.1201/9780203711194 is OK
- 10.1006/jcph.1998.5901 is OK
- 10.1137/S106482759324700X is OK
- 10.1016/j.jcp.2004.12.021 is OK
- 10.1002/fld.1070 is OK
- doi:10.1002/fld.1071 is OK
- 10.1002/qj.1913 is OK
- 10.3233/SPR-140379 is OK
- 10.1016/j.jcp.2015.02.003 is OK
- 10.5194/gmd-8-1005-2015 is OK
- 10.1007/978-3-319-93864-6_5 is OK
- 10.5194/gmd-12-651-2019 is OK
- 10.1016/j.cam.2019.05.023 is OK
- 10.5194/gmd-2020-404 is OK

MISSING DOIs

- None

INVALID DOIs

- None

whedon avatar Nov 08 '21 05:11 whedon

thank you @danielskatz, indeed the URL escapes in the bib file were the cause - good job on the checker side!

slayoo avatar Nov 08 '21 05:11 slayoo

:wave: @dmikushin, please update us on how your review is going (this is an automated reminder).

whedon avatar Nov 21 '21 07:11 whedon

:wave: @olekravchenko, please update us on how your review is going (this is an automated reminder).

whedon avatar Nov 21 '21 07:11 whedon

Hello @dmikushin and @olekravchenko - I was wondering if you needed anything in starting the reviews? I noticed that there hasn't been activity since the review was initiated. I'm happy to help or answer questions in getting started.

mjsottile avatar Dec 03 '21 02:12 mjsottile

Hi @slayoo : it appears that the reviewers who agreed to review have vanished and are unresponsive (unless you’ve heard from them outside this review thread). This unfortunately happens sometimes. I will identify alternative reviewers to replace them so the review doesn’t go idle. Give me a day or so to find people to take their place.

mjsottile avatar Dec 09 '21 10:12 mjsottile

Thank you, @mjsottile. (I haven't heard from the reviewers)

slayoo avatar Dec 09 '21 15:12 slayoo

:wave: @d-chambers Would you be willing to perform a review for this submission to the Journal of Open Source Software? I identified you based on your areas of expertise as listed on the spreadsheet of potential JOSS reviewers. Please let me know if you would be interested. Thank you!

mjsottile avatar Dec 10 '21 03:12 mjsottile

👋 @highlando Would you be willing to perform a review for this submission to the Journal of Open Source Software? I identified you based on your areas of expertise as listed on the spreadsheet of potential JOSS reviewers. Please let me know if you would be interested. Thank you!

mjsottile avatar Dec 10 '21 03:12 mjsottile

Hi @mjsottile, unfortunately this is pretty far outside of my area of expertise. So much so, I don't think I would be able to provide any sort of useful review of the functionality of the software (but it does look like an interesting program).

d-chambers avatar Dec 12 '21 22:12 d-chambers

Hi @slayoo , is there a performance comparison between Numba-based PyMPDATA and any other native code implementations of your choice? That would be really interesting to see!

dmikushin avatar Dec 31 '21 22:12 dmikushin

@dmikushin Thanks for feedback. Very good point on including a performance comparison with other implementations,

We have so far:

  • looked at wall times for these tests which have direct counterparts in libmpdata++ (Github Actions used in both projects - times on similar virtual machines are available for each CI build, but these include compilation times in the case of PyMPDATA, hence not straightforward to analyse "as is", but that would be the first candidate to start the analysis from);
  • analysed execution times of various MPDATA varians versus upwind times for PyMPDATA and corroborated it with such wall-time ratios from literature, see Table 2 here: https://arxiv.org/pdf/2011.14726.pdf

Comparing with libmpdata++ times is certainly doable (February/March?). Any other suggestion on what to compare with?

slayoo avatar Jan 03 '22 00:01 slayoo

wave @highlando Would you be willing to perform a review for this submission to the Journal of Open Source Software? I identified you based on your areas of expertise as listed on the spreadsheet of potential JOSS reviewers. Please let me know if you would be interested. Thank you!

Sorry for my late reply. This is not exactly my area of expertise and I'm well occupied with other reviews right now. So I would prefer to skip this one. Please let me know, if there is an urgent need for a comment. :)

highlando avatar Jan 03 '22 20:01 highlando

@dmikushin Thanks for providing some feedback about this submission. Did you have a chance to look at over entries from the review checklist at the top of this issue?

mjsottile avatar Jan 12 '22 23:01 mjsottile

wave @highlando Would you be willing to perform a review for this submission to the Journal of Open Source Software? I identified you based on your areas of expertise as listed on the spreadsheet of potential JOSS reviewers. Please let me know if you would be interested. Thank you!

Sorry for my late reply. This is not exactly my area of expertise and I'm well occupied with other reviews right now. So I would prefer to skip this one. Please let me know, if there is an urgent need for a comment. :)

@highlando No problem. Thanks for responding!

mjsottile avatar Jan 12 '22 23:01 mjsottile