modin
modin copied to clipboard
PERF-#4777: Don't use `copy=True` parameter for `concat` calls inside `to_pandas`
Signed-off-by: Myachev [email protected]
What do these changes do?
- [ ] commit message follows format outlined here
- [ ] passes
flake8 modin/ asv_bench/benchmarks scripts/doc_checker.py - [ ] passes
black --check modin/ asv_bench/benchmarks scripts/doc_checker.py - [ ] signed commit with
git commit -s - [x] Resolves #4777
- [ ] tests added and passing
- [ ] module layout described at
docs/development/architecture.rstis up-to-date - [ ] added (Issue Number: PR title (PR Number)) and github username to release notes for next major release
Codecov Report
Merging #4778 (5aae430) into master (eee5f43) will increase coverage by
5.23%. The diff coverage is100.00%.
@@ Coverage Diff @@
## master #4778 +/- ##
==========================================
+ Coverage 84.63% 89.86% +5.23%
==========================================
Files 265 266 +1
Lines 19626 19909 +283
==========================================
+ Hits 16610 17892 +1282
+ Misses 3016 2017 -999
| Impacted Files | Coverage Δ | |
|---|---|---|
| ...dataframe/pandas/partitioning/partition_manager.py | 89.25% <ø> (+3.88%) |
:arrow_up: |
| modin/core/dataframe/pandas/utils.py | 100.00% <100.00%> (ø) |
|
| modin/logging/config.py | 94.59% <0.00%> (-1.30%) |
:arrow_down: |
| modin/experimental/batch/test/test_pipeline.py | 92.75% <0.00%> (ø) |
|
| modin/pandas/groupby.py | 93.77% <0.00%> (+0.23%) |
:arrow_up: |
| modin/pandas/series.py | 94.23% <0.00%> (+0.24%) |
:arrow_up: |
| modin/pandas/series_utils.py | 99.43% <0.00%> (+0.56%) |
:arrow_up: |
| ...odin/core/storage_formats/pandas/query_compiler.py | 96.59% <0.00%> (+0.56%) |
:arrow_up: |
| modin/core/io/text/excel_dispatcher.py | 94.01% <0.00%> (+0.85%) |
:arrow_up: |
| modin/experimental/xgboost/xgboost.py | 87.61% <0.00%> (+0.95%) |
:arrow_up: |
| ... and 45 more |
:mega: We’re building smart automated test selection to slash your CI/CD build times. Learn more
Nice!
LGTM
LGTM
Happy to merge but there are the following issues in our CI: ValueError: buffer source array is read-only. I don't have fix for this for now.
CI: ValueError: buffer source array is read-only
does this happen on pandas main too?
CI: ValueError: buffer source array is read-only
does this happen on pandas main too?
I don’t know, I don’t have the right environment for this at hand for now. However, we can definitely try to make at least one copy instead of two (if we can’t get rid of them at all)
Any progress here?
Any progress here?
Unfortunately no progress
I don't know what else I can do here
~Blocked by https://github.com/modin-project/modin/pull/6706~