modin icon indicating copy to clipboard operation
modin copied to clipboard

PERF-#4743: avoid `partition.length()` in the parquet dispatcher

Open anmyachev opened this issue 2 years ago • 1 comments

Signed-off-by: Myachev [email protected]

What do these changes do?

  • [x] commit message follows format outlined here
  • [x] passes flake8 modin/ asv_bench/benchmarks scripts/doc_checker.py
  • [x] passes black --check modin/ asv_bench/benchmarks scripts/doc_checker.py
  • [x] signed commit with git commit -s
  • [x] Resolves #4743
  • [x] tests added and passing
  • [x] module layout described at docs/development/architecture.rst is up-to-date
  • [x] added (Issue Number: PR title (PR Number)) and github username to release notes for next major release

anmyachev avatar Sep 13 '22 14:09 anmyachev

Codecov Report

Merging #4960 (7871c7b) into master (b5f7ed3) will decrease coverage by 0.34%. The diff coverage is 73.85%.

:exclamation: Current head 7871c7b differs from pull request most recent head 122f4d9. Consider uploading reports for the commit 122f4d9 to get more accurate results

@@            Coverage Diff             @@
##           master    #4960      +/-   ##
==========================================
- Coverage   84.91%   84.56%   -0.35%     
==========================================
  Files         266      256      -10     
  Lines       19763    19345     -418     
==========================================
- Hits        16781    16359     -422     
- Misses       2982     2986       +4     
Impacted Files Coverage Δ
modin/_compat/core/py36/pandas_common.py 0.00% <0.00%> (ø)
modin/_compat/pandas_api/latest/window.py 100.00% <ø> (ø)
modin/_compat/pandas_api/py36/__init__.py 0.00% <0.00%> (ø)
modin/_compat/pandas_api/py36/base.py 0.00% <0.00%> (ø)
modin/_compat/pandas_api/py36/dataframe.py 0.00% <0.00%> (ø)
modin/_compat/pandas_api/py36/io.py 0.00% <0.00%> (ø)
modin/_compat/pandas_api/py36/resample.py 0.00% <0.00%> (ø)
modin/_compat/pandas_api/py36/series.py 0.00% <0.00%> (ø)
modin/_compat/pandas_api/py36/window.py 0.00% <ø> (ø)
...tations/pandas_on_python/partitioning/partition.py 90.00% <0.00%> (ø)
... and 64 more

:mega: We’re building smart automated test selection to slash your CI/CD build times. Learn more

codecov[bot] avatar Sep 13 '22 14:09 codecov[bot]

@anmyachev could you rebase the PR so we can re-run CI and make sure there are no issues here?

done

anmyachev avatar Sep 27 '22 15:09 anmyachev

I'm curious about the performance implications here. Is there a way you could check quickly @anmyachev ?

This is part of the change required for asynchronous execution. The performance difference should not be visible now.

Without knowing that this is needed for asynchronous execution, these changes can be considered as refactoring, since there is no immediate impact on performance. I can change the first commit category (to REFACTOR) if it's better.

anmyachev avatar Sep 27 '22 16:09 anmyachev

We should label things PERF if they're the PR which helps performance. If they're building the base they could be FEAT or REFACTOR, but not PERF.

vnlitvinov avatar Oct 05 '22 16:10 vnlitvinov

We should label things PERF if they're the PR which helps performance. If they're building the base they could be FEAT or REFACTOR, but not PERF.

Apparently yes, I was wrong about that. Convert to draft.

anmyachev avatar Oct 05 '22 21:10 anmyachev

not actual

anmyachev avatar Mar 01 '23 00:03 anmyachev