heat icon indicating copy to clipboard operation
heat copied to clipboard

Expand distributed indexing, match numpy indexing scheme

Open ClaudiaComito opened this issue 2 years ago • 4 comments

Description

DRAFT

Intro: main points from Numpy array indexing scheme

Heat: indexing massive, memory-distributed arrays

process-local indexing

distributed indexing

  • sorted key
  • non-sorted key

Memory footprint

Scaling behaviour

Issue/s resolved: #914 #918

Changes proposed:

Type of change

Memory requirements

Performance

Due Diligence

  • [ ] All split configurations tested
  • [ ] Multiple dtypes tested in relevant functions
  • [ ] Documentation updated (if needed)
  • [ ] Updated changelog.md under the title "Pending Additions"

Does this change modify the behaviour of other functions? If so, which?

yes / no

skip ci

ClaudiaComito avatar Mar 24 '22 05:03 ClaudiaComito

👇 Click on the image for a new way to code review

Review these changes using an interactive CodeSee Map

Legend

CodeSee Map legend

ghost avatar Jul 26 '23 10:07 ghost

just a comment: in the fft-module (if already merged at time merging this PR) some commented-out parts of test_rfftn_irfftn can be added again if this PR is ready

mrfh92 avatar Oct 27 '23 15:10 mrfh92

Codecov Report

Attention: Patch coverage is 30.31915% with 131 lines in your changes are missing coverage. Please review.

Project coverage is 66.88%. Comparing base (8ba63c5) to head (5995639).

:exclamation: Current head 5995639 differs from pull request most recent head a8f2d57. Consider uploading reports for the commit a8f2d57 to get more accurate results

Files Patch % Lines
heat/core/dndarray.py 26.70% 129 Missing :warning:
heat/core/indexing.py 83.33% 2 Missing :warning:
Additional details and impacted files
@@             Coverage Diff             @@
##             main     #938       +/-   ##
===========================================
- Coverage   91.92%   66.88%   -25.04%     
===========================================
  Files          79       65       -14     
  Lines       11721    10017     -1704     
===========================================
- Hits        10774     6700     -4074     
- Misses        947     3317     +2370     
Flag Coverage Δ
unit 66.88% <30.31%> (-25.04%) :arrow_down:

Flags with carried forward coverage won't be shown. Click here to find out more.

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.

codecov[bot] avatar Nov 27 '23 03:11 codecov[bot]