pint icon indicating copy to clipboard operation
pint copied to clipboard

Defer loading modules used only in `compat.upcast_types`

Open mattwthompson opened this issue 2 years ago • 2 comments

Work in progress. I will rebase #1491 or something similar into this as well.

In [1]: >>> import sys
   ...: >>> from pint import UnitRegistry
   ...: >>> 'pandas' in sys.modules
Out[1]: False

A quick benchmark (will update when tests are fixed) with pandas installed in this environment:

$ # origin/master
$ hyperfine 'python -c "import pint"'
Benchmark 1: python -c "import pint"
  Time (mean ± σ):      1.589 s ±  0.118 s    [User: 1.550 s, System: 0.323 s]
  Range (min … max):    1.492 s …  1.867 s    10 runs

$ git checkout fix-1460
$ hyperfine 'python -c "import pint"'Benchmark 1: python -c "import pint"
  Time (mean ± σ):     672.5 ms ±  37.3 ms    [User: 869.0 ms, System: 135.8 ms]
  Range (min … max):   643.3 ms … 762.3 ms    10 runs

The headline issue is resolved but I seem to have broken many things along the way; removing upcast_types will break other tools that rely on it existing as a list pre-populated with all types it can import.

  • [x] Closes #1460
  • [x] Executed pre-commit run --all-files with no errors
  • [ ] The change is fully covered by automated unit tests
  • [ ] Documented in docs/ as appropriate
  • [ ] Added an entry to the CHANGES file

mattwthompson avatar Mar 30 '22 17:03 mattwthompson

Thanks can you provide some benchmark with import time if you can along with this PR.

Just a %timeit running import should be fine

jules-ch avatar Mar 30 '22 18:03 jules-ch

I have a couple tests passing locally, but the import time drops from about 320 to 160 ms:

(pint-dev) [pint] git checkout upstream/master                                                        16:30:24  ☁  208d36a ☀
HEAD is now at 208d36a Merge pull request #1569 from edouardberthe/patch-1
(pint-dev) [pint] hyperfine --warmup 10 'python -c "import pint"'                                     16:30:25  ☁  208d36a ☀

Benchmark 1: python -c "import pint"
  Time (mean ± σ):     320.2 ms ±   6.5 ms    [User: 689.1 ms, System: 266.8 ms]
  Range (min … max):   311.8 ms … 331.4 ms    10 runs

(pint-dev) [pint] git checkout fix-1460                                                               16:30:34  ☁  208d36a ☀
Previous HEAD position was 208d36a Merge pull request #1569 from edouardberthe/patch-1
Switched to branch 'fix-1460'
Your branch is up to date with 'origin/fix-1460'.
(pint-dev) [pint] hyperfine --warmup 10 'python -c "import pint"'                                    16:30:38  ☁  fix-1460 ☀

Benchmark 1: python -c "import pint"
  Time (mean ± σ):     159.5 ms ±   2.8 ms    [User: 209.7 ms, System: 72.2 ms]
  Range (min … max):   152.3 ms … 163.2 ms    18 runs

mattwthompson avatar Sep 19 '22 21:09 mattwthompson

Thanks!

mattwthompson avatar Apr 25 '23 02:04 mattwthompson

Thank you! (I am fixing a test but it is worth it)

hgrecco avatar Apr 25 '23 02:04 hgrecco

I'm not entirely sure, but I think this change has broken the compatibility with pint-pandas 0.3.

After creating a new virtual environment and installing both pint and pint-pandas, I cannot import the latter.

python3.9 -m venv venv
source venv/bin/activate
python --version
python -m pip install --upgrade pip
python -m pip install pint pint-pandas
python -c "import pint_pandas"

Results in:

Python 3.9.11
...
Successfully installed numpy-1.24.3 pandas-2.0.1 pint-0.21 pint-pandas-0.3 python-dateutil-2.8.2 pytz-2023.3 six-1.16.0 tzdata-2023.3
...
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/Users/anritter/Downloads/venv/lib/python3.9/site-packages/pint_pandas/__init__.py", line 3, in <module>
    from .pint_array import PintArray, PintType
  File "/Users/anritter/Downloads/venv/lib/python3.9/site-packages/pint_pandas/pint_array.py", line 995, in <module>
    compat.upcast_types.append(PintArray)
AttributeError: module 'pint.compat' has no attribute 'upcast_types'

The same also happens with Python 3.10.11 and Python 3.11.3.

aaaandi avatar May 09 '23 22:05 aaaandi

Please follow https://github.com/hgrecco/pint-pandas/issues/168 for progress on that issue

mattwthompson avatar May 09 '23 22:05 mattwthompson