Replace many Pandas operations with NumPy
Describe your changes
- use numpy instead of Pandas to avoid the overhead
Perf before:
320361688 function calls (316213771 primitive calls) in 116.715 seconds
Perf after:
79419311 function calls (78769517 primitive calls) in 43.085 seconds
which is more or less in-line with the circular shapes.
Checklist
- [x] Test cases have been modified/added to cover any code changes.
- [x] Docstrings have been modified/created for any code changes.
- [x] All linting and formatting checks pass (see the contributing guidelines for more information).
Thanks for the PR, @JCGoran! As I'm sure you've seen, I have a backlog to get through :smile: I hope to get to this in the next few weeks.
Codecov Report
All modified and coverable lines are covered by tests :white_check_mark:
Project coverage is 98.53%. Comparing base (
e440ee7) to head (0a25272).
Additional details and impacted files
@@ Coverage Diff @@
## main #198 +/- ##
=======================================
Coverage 98.53% 98.53%
=======================================
Files 58 58
Lines 1907 1915 +8
Branches 114 114
=======================================
+ Hits 1879 1887 +8
Misses 25 25
Partials 3 3
| Files with missing lines | Coverage Δ | |
|---|---|---|
| src/data_morph/data/dataset.py | 74.07% <100.00%> (+0.65%) |
:arrow_up: |
| src/data_morph/data/stats.py | 100.00% <100.00%> (ø) |
|
| src/data_morph/morpher.py | 100.00% <100.00%> (ø) |
|
| src/data_morph/plotting/static.py | 100.00% <100.00%> (ø) |
|
| tests/data/test_stats.py | 100.00% <100.00%> (ø) |
|
| tests/test_morpher.py | 100.00% <100.00%> (ø) |
- Flaky Tests Detection - Detect and resolve failed and flaky tests
Bump, this is more or less ready for review as-is.
I haven't forgotten 😄 I'm going to work through the PyCon Taiwan sprint PRs first since I couldn't get to them all at the event, and I want to think more about the design of the internals here. I'm traveling right now and will have very limited time for the next couple of weeks.