modin
modin copied to clipboard
PERF-#4743: avoid `partition.length()` in the parquet dispatcher
Signed-off-by: Myachev [email protected]
What do these changes do?
- [x] commit message follows format outlined here
- [x] passes
flake8 modin/ asv_bench/benchmarks scripts/doc_checker.py
- [x] passes
black --check modin/ asv_bench/benchmarks scripts/doc_checker.py
- [x] signed commit with
git commit -s
- [x] Resolves #4743
- [x] tests added and passing
- [x] module layout described at
docs/development/architecture.rst
is up-to-date - [x] added (Issue Number: PR title (PR Number)) and github username to release notes for next major release
Codecov Report
Merging #4960 (7871c7b) into master (b5f7ed3) will decrease coverage by
0.34%
. The diff coverage is73.85%
.
:exclamation: Current head 7871c7b differs from pull request most recent head 122f4d9. Consider uploading reports for the commit 122f4d9 to get more accurate results
@@ Coverage Diff @@
## master #4960 +/- ##
==========================================
- Coverage 84.91% 84.56% -0.35%
==========================================
Files 266 256 -10
Lines 19763 19345 -418
==========================================
- Hits 16781 16359 -422
- Misses 2982 2986 +4
Impacted Files | Coverage Δ | |
---|---|---|
modin/_compat/core/py36/pandas_common.py | 0.00% <0.00%> (ø) |
|
modin/_compat/pandas_api/latest/window.py | 100.00% <ø> (ø) |
|
modin/_compat/pandas_api/py36/__init__.py | 0.00% <0.00%> (ø) |
|
modin/_compat/pandas_api/py36/base.py | 0.00% <0.00%> (ø) |
|
modin/_compat/pandas_api/py36/dataframe.py | 0.00% <0.00%> (ø) |
|
modin/_compat/pandas_api/py36/io.py | 0.00% <0.00%> (ø) |
|
modin/_compat/pandas_api/py36/resample.py | 0.00% <0.00%> (ø) |
|
modin/_compat/pandas_api/py36/series.py | 0.00% <0.00%> (ø) |
|
modin/_compat/pandas_api/py36/window.py | 0.00% <ø> (ø) |
|
...tations/pandas_on_python/partitioning/partition.py | 90.00% <0.00%> (ø) |
|
... and 64 more |
:mega: We’re building smart automated test selection to slash your CI/CD build times. Learn more
@anmyachev could you rebase the PR so we can re-run CI and make sure there are no issues here?
done
I'm curious about the performance implications here. Is there a way you could check quickly @anmyachev ?
This is part of the change required for asynchronous execution. The performance difference should not be visible now.
Without knowing that this is needed for asynchronous execution, these changes can be considered as refactoring, since there is no immediate impact on performance. I can change the first commit category (to REFACTOR) if it's better.
We should label things PERF
if they're the PR which helps performance. If they're building the base they could be FEAT
or REFACTOR
, but not PERF
.
We should label things
PERF
if they're the PR which helps performance. If they're building the base they could beFEAT
orREFACTOR
, but notPERF
.
Apparently yes, I was wrong about that. Convert to draft.
not actual