data-morph icon indicating copy to clipboard operation
data-morph copied to clipboard

Replace many Pandas operations with NumPy

Open JCGoran opened this issue 1 year ago • 4 comments

Describe your changes

  • use numpy instead of Pandas to avoid the overhead

Perf before:

320361688 function calls (316213771 primitive calls) in 116.715 seconds

Perf after:

79419311 function calls (78769517 primitive calls) in 43.085 seconds

which is more or less in-line with the circular shapes.

Checklist

  • [x] Test cases have been modified/added to cover any code changes.
  • [x] Docstrings have been modified/created for any code changes.
  • [x] All linting and formatting checks pass (see the contributing guidelines for more information).

JCGoran avatar Jul 15 '24 22:07 JCGoran

Thanks for the PR, @JCGoran! As I'm sure you've seen, I have a backlog to get through :smile: I hope to get to this in the next few weeks.

stefmolin avatar Jul 16 '24 22:07 stefmolin

Codecov Report

All modified and coverable lines are covered by tests :white_check_mark:

Project coverage is 98.53%. Comparing base (e440ee7) to head (0a25272).

Additional details and impacted files

Impacted file tree graph

@@           Coverage Diff           @@
##             main     #198   +/-   ##
=======================================
  Coverage   98.53%   98.53%           
=======================================
  Files          58       58           
  Lines        1907     1915    +8     
  Branches      114      114           
=======================================
+ Hits         1879     1887    +8     
  Misses         25       25           
  Partials        3        3           
Files with missing lines Coverage Δ
src/data_morph/data/dataset.py 74.07% <100.00%> (+0.65%) :arrow_up:
src/data_morph/data/stats.py 100.00% <100.00%> (ø)
src/data_morph/morpher.py 100.00% <100.00%> (ø)
src/data_morph/plotting/static.py 100.00% <100.00%> (ø)
tests/data/test_stats.py 100.00% <100.00%> (ø)
tests/test_morpher.py 100.00% <100.00%> (ø)
---- 🚨 Try these New Features:

codecov[bot] avatar Jul 16 '24 22:07 codecov[bot]

Bump, this is more or less ready for review as-is.

JCGoran avatar Sep 24 '24 16:09 JCGoran

I haven't forgotten 😄 I'm going to work through the PyCon Taiwan sprint PRs first since I couldn't get to them all at the event, and I want to think more about the design of the internals here. I'm traveling right now and will have very limited time for the next couple of weeks.

stefmolin avatar Sep 25 '24 12:09 stefmolin