qcodes import takes a long time

[eendebakpt]

Importing qcodes with import qcodes takes a long time (> 2 seconds on the system tested). The import time can be analysed with tools like https://github.com/asottile/importtime-waterfall

Several components could perhaps not be loaded by default, or with a lazy loader (e.g. https://scientific-python.org/specs/spec-0001/, but there are more tools)

An example is the import from opencensus.ext.azure.log_exporter import AzureLogHandler from qcodes.logger.logger. By default qc.config.telemetry.enabled is False, so the import is executed at startup, but not used.

(more examples can be found by analysing the output of importtime-waterfall)

[jenshnielsen]

@eendebakpt Which version of qcodes have you tested. I spend a fair amount of time improving this before 0.32 for example. For example the imports of opencensus azure are lazy and has been so since 0.32. See https://github.com/QCoDeS/Qcodes/commit/9f90a67607558689786e347665a2d2e9b1dad1bf

[eendebakpt]

@jenshnielsen With qcodes updated to master the import of azure is indeed lazy. There is still room for improvement though.

Running:

python -c "import time; import xarray; import pandas; import scipy; t0=time.perf_counter(); import qcodes; dt=time.perf_counter()-t0; print(f'{1e3*dt:.2f} ms')"

results in a > 1 second import time of qcodes on my system.

Some more candidates to reduce the import time:

• From the qcodes init:

from qcodes.instrument_drivers.test import test_instrument, test_instruments

Can be removed, or made lazy (for example with a mechanism like https://github.com/sympy/sympy/pull/23832). Tests should not impact runtime import performances.

• qcodes.data.hdf5_format. This is part of the legacy dataset, but still imported via

https://github.com/QCoDeS/Qcodes/blob/master/qcodes/parameters/array_parameter.py#L10

• The whole legacy dataset module is imported via

https://github.com/QCoDeS/Qcodes/blob/master/qcodes/dataset/init.py#L27

(this triggers also imports of xarray and scipy, maybe they would have been imported anyway)

• The qcodes.monitor triggers websockets. Perhaps import of qcodes.monitor can be removed from the qcodes.__init__

• There are some checks to prevent to import of pandas (e.g. https://github.com/QCoDeS/Qcodes/blob/eef1cf42ca9f966031c7a27a60ad10052518a993/qcodes/dataset/exporters/export_to_pandas.py#L9-L10), but pandas is imported anyway (for me on this line: https://github.com/QCoDeS/Qcodes/blob/652ec5404a19f7456cf8c2e05ff4c62c2161ffe1/qcodes/logger/log_analysis.py#L8)

[eendebakpt]

Closing as the import time was improved in two pr's