Measuring import times

[njzjz]

I have a package and I hope the first import times are minimal. I used subprocess.check_output to start a new Python process and import modules. However, I just learned that system calls are not included in the total time, so Popen.communicate is fully ignored, as shown below. Is there a recommended way to test the first import time?

[art049]

We added a sample https://github.com/CodSpeedHQ/pytest-codspeed/pull/73 measuring the import performance.

The trick is to mock sys.modules to make sure the module is not cached by CPython when running multiple times.

Can you give it a try for your use case ?

[njzjz]

Thanks. It works for most cases but does not work for numpy:

      import numpy as np
  /opt/hostedtoolcache/Python/3.12.9/x64/lib/python3.12/site-packages/numpy/__init__.py:114: in <module>
      from numpy.__config__ import show_config
  /opt/hostedtoolcache/Python/3.12.9/x64/lib/python3.12/site-packages/numpy/__config__.py:4: in <module>
      from numpy._core._multiarray_umath import (
  /opt/hostedtoolcache/Python/3.12.9/x64/lib/python3.12/site-packages/numpy/_core/__init__.py:23: in <module>
      from . import multiarray
  /opt/hostedtoolcache/Python/3.12.9/x64/lib/python3.12/site-packages/numpy/_core/multiarray.py:10: in <module>
      from . import overrides
  _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
  
      """Implementation of __array_function__ overrides from NEP-18."""
      import collections
      import functools
      
      from .._utils import set_module
      from .._utils._inspect import getargspec
  >   from numpy._core._multiarray_umath import (
          add_docstring,  _get_implementing_args, _ArrayFunctionDispatcher)
  E   KeyError: 'numpy.exceptions'

[njzjz]

The new walltime instrument could resolve this issue. https://codspeed.io/docs/instruments/walltime