mdanalysis icon indicating copy to clipboard operation
mdanalysis copied to clipboard

'MDAnalysis.analysis.density' parallelization

Open talagayev opened this issue 1 year ago • 5 comments

Fixes #4677 attempt

Changes made in this Pull Request:

  • added backends and aggregators to DensityAnalysis in analysis.density
  • added client_DensityAnalysis in conftest.py
  • added client_DensityAnalysis to the tests in test_density.py

Here again I am not sure for this case, since the Error message:

AttributeError: 'DensityAnalysis' object has no attribute '_grid' appears in the tests and I am not sure if moving _grid and all the code necessary for it from _prepare to __init__ will fix the case like it was the case in #4718

PR Checklist

  • [x] Tests?
  • [ ] Docs?
  • [ ] CHANGELOG updated?
  • [x] Issue raised/referenced?

Developers certificate of origin


📚 Documentation preview 📚: https://mdanalysis--4729.org.readthedocs.build/en/4729/

talagayev avatar Oct 08 '24 21:10 talagayev

Hello @talagayev! Thanks for updating this PR. We checked the lines you've touched for PEP 8 issues, and found:

Line 426:66: W291 trailing whitespace Line 427:68: W291 trailing whitespace Line 428:36: W291 trailing whitespace

Comment last updated at 2024-12-18 19:05:59 UTC

pep8speaks avatar Oct 08 '24 21:10 pep8speaks

Linter Bot Results:

Hi @talagayev! Thanks for making this PR. We linted your code and found the following:

Some issues were found with the formatting of your code.

Code Location Outcome
main package ⚠️ Possible failure
testsuite ⚠️ Possible failure

Please have a look at the darker-main-code and darker-test-code steps here for more details: https://github.com/MDAnalysis/mdanalysis/actions/runs/11265261686/job/31326801780


Please note: The black linter is purely informational, you can safely ignore these outcomes if there are no flake8 failures!

github-actions[bot] avatar Oct 08 '24 21:10 github-actions[bot]

Codecov Report

All modified and coverable lines are covered by tests :white_check_mark:

Project coverage is 93.63%. Comparing base (a3672f2) to head (506597d). Report is 44 commits behind head on develop.

Additional details and impacted files
@@             Coverage Diff             @@
##           develop    #4729      +/-   ##
===========================================
- Coverage    93.65%   93.63%   -0.03%     
===========================================
  Files          177      189      +12     
  Lines        21774    22845    +1071     
  Branches      3064     3064              
===========================================
+ Hits         20393    21391     +998     
- Misses         929     1002      +73     
  Partials       452      452              

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

codecov[bot] avatar Oct 10 '24 00:10 codecov[bot]

Three aspects in my fix.

  • Attributes (private or not) that will be used later in _conclude should be defined in __init__, not in _prepare. @marinegor, I think it would be helpful to explicitly mention this in the documentation. Additionally, while the illustrative figure (https://docs.mdanalysis.org/dev/documentation_pages/analysis/parallelization.html#split-apply-combine) suggests that _prepare occurs before splitting, this is not the case.

  • The _edges and similar attributes must be defined in __init__ by nature because they need to be consistent across all workers and are determined based on the current positions of the atomgroup (https://github.com/MDAnalysis/mdanalysis/pull/4729/files#diff-28b7c110c6231f9f2ea864bbdecb9cb485edbed494d7e266d37896bde7b50b0dR421) .

  • The results being aggregated are actually _grid, not DensityDensity serves more as a postprocessing step that summarizes the information.

yuxuanzhuang avatar Oct 10 '24 00:10 yuxuanzhuang

@yuxuanzhuang could you please raise issues for the updates to the parallelization documentation (https://github.com/MDAnalysis/mdanalysis/pull/4729#issuecomment-2403681751) — I think that's very important to make clear, given that we are now repeatedly running in these issues. Thank you!

orbeckst avatar Oct 22 '24 16:10 orbeckst

@yuxuanzhuang thanks, the illustration is indeed wrong, as well as the accompanying text. I fixed them in #4760

marinegor avatar Oct 24 '24 20:10 marinegor