sympy
sympy copied to clipboard
Published
20 hours ago •
sympy
sympy
Improve sympy import performance by lazy loading of testing code
Brief description of what is fixed or changed
The import waterfall of sympy shows 594 modules imported (command importtime-waterfall sympy). The total import time (tested with /bin/test_import) is
Number of tests: 50
The speed of "import sympy" is: 0.272500 +- 0.003218
Performance of the sympy import can be improved if the number of imported modules is reduced.
This PR reduced the import time by
- Using a lazy import of
sympy.testing.runtestsinsympy.utilities.decorator. - Creating lazy wrappers for
sympy.testing.testandsympy.testing.doctestin the mainsympymodule.
The lazy loading eliminates the import of sympy.testing.test, which triggers loading of unittest and asyncio.
The new import time is:
Number of tests: 50
The speed of "import sympy" is: 0.243919 +- 0.003072
The docstrings of test and doctest are now quite sparse. We could copy the docstrings of the corresponding methods in sympy.testing.runtests, or we could deprecate the methods in the main sympy module.
Other comments
This PR was written to be backwards compatible. To further reduce the import time, one could consider to not load certain modules by default. For example sympy.printing.mathematica could be changed to only load when a user explictly calls import sympy.printing.mathematica
Details of the benchmarks:
[MASTER] Result of python bin/test_import
Note: the first run (warm up) was not included in the average + std dev
All runs (including warm up):
[0.2796212559997002, 0.2781042780002281, 0.27615861999993285, 0.2748804050002036, 0.2746562840002298, 0.2732063799999196, 0.27364490100035255, 0.2708097899999302, 0.2752684740003133, 0.270890703999612, 0.2738728619997346, 0.26944935499977873, 0.2704215500002647, 0.27218362399980833, 0.2721846950003055, 0.2744498849997399, 0.26823755899977186, 0.27238844600015, 0.2831268329996419, 0.2742866089997733, 0.27588116900005843, 0.281494990000283, 0.27313322700001663, 0.268633022999893, 0.2680984220000937, 0.27662726300013674, 0.2734363519998624, 0.275800481999795, 0.27083820200004993, 0.2759205409997776, 0.2724356540002191, 0.2689656769998692, 0.272148050999931, 0.2699153190001198, 0.26858375200026785, 0.27064090600015334, 0.2707692060002955, 0.2712384749997909, 0.2706845650000105, 0.2712465599997813, 0.27598768999996537, 0.26832291200025793, 0.2695224099998086, 0.2709240030003457, 0.26965993800013166, 0.27197772999988956, 0.2692769809996207, 0.2698648979999234, 0.2708809119999387, 0.2736668140000802, 0.27018944299970826]
Number of tests: 50
The speed of "import sympy" is: 0.272500 +- 0.003218
[PR] Result of python bin/test_import
Note: the first run (warm up) was not included in the average + std dev
All runs (including warm up):
[0.24775494799996522, 0.24446511799988002, 0.24558605499987607, 0.24339464699960445, 0.24083171199981734, 0.24163932600004046, 0.24593805700033045, 0.25555351100001644, 0.2405642789999547, 0.24108691999981602, 0.2436481409999942, 0.24590180000041073, 0.24222797199990964, 0.2418284290001793, 0.24650318899966805, 0.2435498390000248, 0.24492248100023062, 0.24199547499983964, 0.24216606300024068, 0.24247888899981263, 0.24273960599975908, 0.24888989000010042, 0.2411293659997682, 0.26128544099992723, 0.24476161800021146, 0.24317696499974772, 0.25251023200007694, 0.24237984000001234, 0.24224764900009177, 0.2431937559999824, 0.2482696600000054, 0.24661963899961847, 0.24343341199983115, 0.24264238399973692, 0.24271199299982982, 0.24021964699977616, 0.24562413000012384, 0.24461949899978208, 0.2423081650003951, 0.240376594999816, 0.242632059999778, 0.24419342999999571, 0.24679355099988243, 0.242847428999994, 0.24469787399993947, 0.24236903000019083, 0.2395998730003157, 0.24556445400003213, 0.24200651399996786, 0.24416150100023515, 0.24275182900009895]
Number of tests: 50
The speed of "import sympy" is: 0.244221 +- 0.003791
Result of importtime-waterfall sympy
sympy (1043)
mpmath (1501)
mpmath.usertools (161)
mpmath.ctx_fp (306)
mpmath.ctx_base (383)
mpmath.libmp.backend (29)
mpmath.libmp (302)
mpmath.libmp.libmpf (718)
math (208)
bisect (146)
_bisect (32)
mpmath.libmp.backend (197)
gmpy2 (116)
gmpy (120)
mpmath.libmp.libintmath (489)
mpmath.libmp.libmpc (336)
mpmath.libmp.libelefun (288)
mpmath.libmp.libhyper (274)
mpmath.libmp.gammazeta (844)
mpmath.libmp.libmpi (241)
mpmath.functions.functions (16)
mpmath.functions (440)
mpmath.functions.functions (336)
cmath (52)
mpmath.functions.factorials (169)
mpmath.functions.hypergeometric (269)
mpmath.functions.expintegrals (168)
mpmath.functions.bessel (299)
mpmath.functions.orthogonal (174)
mpmath.functions.theta (145)
mpmath.functions.elliptic (195)
mpmath.functions.zeta (320)
__future__ (156)
mpmath.functions.rszeta (195)
mpmath.functions.zetazeros (261)
mpmath.functions.qfunctions (134)
mpmath.calculus.quadrature (347)
mpmath.calculus (252)
mpmath.calculus.calculus (151)
mpmath.calculus.approximation (134)
mpmath.calculus.differentiation (169)
mpmath.calculus.extrapolation (284)
mpmath.calculus.polynomials (135)
mpmath.calculus.inverselaplace (218)
mpmath.calculus.optimization (298)
copy (205)
weakref (387)
_weakrefset (175)
copyreg (164)
org.python.core (23)
org.python (25)
org (118)
mpmath.calculus.odes (163)
mpmath.matrices.matrices (248)
mpmath.matrices (174)
mpmath.matrices.eigen (194)
mpmath.matrices.eigen_symmetric (187)
mpmath.matrices.calculus (150)
mpmath.matrices.linalg (184)
mpmath.identification (274)
mpmath.visualization (201)
colorsys (128)
mpmath.math2 (241)
mpmath.function_docs (402)
mpmath.ctx_mp (1505)
re (496)
enum (598)
sre_compile (285)
_sre (71)
sre_parse (408)
sre_constants (257)
mpmath.rational (216)
numbers (422)
mpmath.ctx_mp_python (2099)
mpmath.ctx_iv (357)
sympy.release (210)
sympy.core (304)
sympy.core.sympify (332)
typing (1328)
collections.abc (150)
inspect (1344)
dis (388)
opcode (305)
_opcode (144)
linecache (163)
tokenize (797)
token (155)
string (691)
_string (38)
sympy.core.random (244)
sympy.utilities.iterables (20)
sympy.utilities (191)
sympy.utilities.iterables (462)
sympy.utilities.enumerative (182)
sympy.utilities.misc (223)
struct (188)
_struct (61)
textwrap (1060)
sympy.utilities.decorator (219)
sympy.testing.runtests (30)
sympy.testing (135)
sympy.testing.runtests (929)
platform (2244)
traceback (252)
pdb (497)
cmd (196)
bdb (356)
fnmatch (141)
code (216)
codeop (157)
glob (302)
pprint (252)
signal (769)
timeit (231)
gc (47)
doctest (1866)
difflib (551)
unittest (261)
unittest.result (250)
unittest.util (230)
unittest.async_case (192)
asyncio (273)
asyncio.base_events (793)
concurrent.futures (504)
concurrent (127)
concurrent.futures._base (462)
logging (1788)
threading (431)
atexit (38)
socket (1472)
_socket (145)
selectors (535)
select (56)
errno (71)
subprocess (429)
msvcrt (179)
_posixsubprocess (50)
ssl (2423)
_ssl (1403)
base64 (333)
binascii (65)
asyncio.constants (211)
asyncio.coroutines (228)
asyncio.base_futures (143)
asyncio.format_helpers (121)
asyncio.log (111)
asyncio.events (986)
contextvars (128)
_contextvars (119)
asyncio.exceptions (187)
_asyncio (245)
asyncio.base_tasks (138)
asyncio.futures (214)
asyncio.protocols (178)
asyncio.sslproto (329)
asyncio.transports (330)
asyncio.staggered (347)
asyncio.locks (259)
asyncio.tasks (295)
asyncio.trsock (238)
asyncio.runners (143)
asyncio.queues (213)
asyncio.streams (281)
asyncio.subprocess (209)
asyncio.unix_events (674)
asyncio.base_subprocess (234)
asyncio.selector_events (351)
unittest.case (685)
unittest.suite (283)
unittest.loader (565)
unittest.main (226)
argparse (701)
gettext (894)
locale (621)
unittest.runner (273)
unittest.signals (135)
random (397)
_sha512 (46)
_random (46)
shutil (541)
zlib (163)
bz2 (267)
_compression (190)
_bz2 (236)
lzma (255)
_lzma (236)
pwd (61)
grp (51)
tempfile (387)
sympy.core.cache (184)
sympy.external (151)
sympy.external.importtools (315)
sympy.external.gmpy (261)
gmpy2 (130)
sympy.external.pythonmpq (211)
decimal (141)
_decimal (679)
fractions (763)
sympy.utilities.exceptions (188)
sympy.utilities.lambdify (385)
sympy.utilities.source (133)
sympy.utilities.timeutils (129)
sympy.core.parameters (158)
sympy.core.basic (764)
sympy.core.assumptions (28696)
sympy.core.facts (239)
sympy.core.logic (850)
sympy.core.core (171)
sympy.core.sorting (220)
sympy.core.kind (272)
sympy.multipledispatch.dispatcher (19)
sympy.multipledispatch (113)
sympy.multipledispatch.core (175)
sympy.multipledispatch.dispatcher (249)
sympy.multipledispatch.conflict (158)
sympy.multipledispatch.utils (105)
sympy.core._print_helpers (132)
sympy.core.singleton (173)
sympy.core.traversal (180)
sympy.core.expr (1325)
sympy.core.evalf (477)
sympy.core.decorators (156)
sympy.core.mul (644)
sympy.core.operations (505)
sympy.core.numbers (4737)
sympy.core.containers (435)
sympy.core.power (572)
sympy.core.function (1519)
sympy.core.add (525)
sympy.core.rules (135)
sympy.core.symbol (895)
sympy.logic.boolalg (19)
sympy.logic (165)
sympy.logic.boolalg (2942)
sympy.logic.inference (235)
sympy.core.relational (1636)
sympy.core.mod (333)
sympy.core.exprtools (409)
sympy.core.coreerrors (135)
sympy.core.multidimensional (175)
sympy.assumptions (156)
sympy.assumptions.assume (525)
sympy.assumptions.ask (549)
sympy.assumptions.cnf (250)
sympy.assumptions.ask_generated (147)
sympy.assumptions.refine (156)
sympy.assumptions.relation (150)
sympy.assumptions.relation.binrel (397)
sympy.polys (324)
sympy.polys.polytools (1382)
sympy.polys.polyoptions (1415)
sympy.polys.polyerrors (426)
sympy.polys.constructor (178)
sympy.polys.domains (410)
sympy.polys.domains.domain (362)
sympy.polys.domains.domainelement (119)
sympy.polys.orderings (237)
sympy.polys.polyutils (325)
sympy.polys.domains.finitefield (226)
sympy.polys.domains.field (226)
sympy.polys.domains.ring (142)
sympy.polys.domains.modularinteger (192)
sympy.polys.domains.simpledomain (139)
sympy.polys.domains.groundtypes (172)
sympy.polys.domains.pythonrational (106)
sympy.polys.domains.integerring (244)
sympy.polys.domains.characteristiczero (136)
sympy.polys.domains.rationalfield (185)
sympy.polys.domains.algebraicfield (250)
sympy.polys.polyclasses (727)
sympy.polys.densebasic (247)
sympy.polys.monomials (275)
sympy.polys.densearith (242)
sympy.polys.densetools (214)
sympy.polys.euclidtools (288)
sympy.polys.galoistools (293)
sympy.polys.polyconfig (157)
sympy.polys.sqfreetools (168)
sympy.polys.factortools (752)
sympy.polys.rootisolation (413)
sympy.polys.domains.gaussiandomains (343)
sympy.polys.domains.realfield (265)
sympy.polys.domains.mpelements (208)
sympy.polys.domains.complexfield (302)
sympy.polys.domains.polynomialring (224)
sympy.polys.domains.compositedomain (124)
sympy.polys.domains.fractionfield (187)
sympy.polys.domains.expressiondomain (240)
sympy.polys.domains.expressionrawdomain (166)
sympy.polys.domains.pythonfinitefield (166)
sympy.polys.domains.pythonintegerring (149)
sympy.polys.domains.gmpyfinitefield (164)
sympy.polys.domains.gmpyintegerring (149)
sympy.polys.domains.pythonrationalfield (219)
sympy.polys.domains.gmpyrationalfield (170)
sympy.polys.fglmtools (167)
sympy.polys.groebnertools (279)
sympy.polys.rationaltools (128)
sympy.polys.polyfuncs (207)
sympy.polys.specialpolys (231)
sympy.ntheory (269)
sympy.ntheory.generate (474)
array (60)
sympy.ntheory.primetest (162)
sympy.ntheory.factor_ (1402)
sympy.ntheory.digits (126)
sympy.ntheory.ecm (201)
sympy.ntheory.partitions_ (235)
sympy.ntheory.residue_ntheory (384)
sympy.ntheory.multinomial (127)
sympy.ntheory.continued_fraction (143)
sympy.ntheory.egyptian_fraction (133)
sympy.ntheory.qs (296)
sympy.polys.rings (595)
sympy.polys.compatibility (772)
sympy.polys.heuristicgcd (140)
sympy.printing.defaults (20)
sympy.printing (430)
sympy.printing.pretty (208)
sympy.printing.pretty.pretty (902)
sympy.printing.conventions (287)
sympy.printing.precedence (199)
sympy.printing.printer (239)
sympy.printing.str (572)
sympy.printing.pretty.stringpict (248)
sympy.printing.pretty.pretty_symbology (596)
unicodedata (55)
sympy.core.alphabets (139)
sympy.printing.latex (1778)
sympy.tensor.array (22)
sympy.tensor (199)
sympy.tensor.indexed (744)
sympy.functions.special.tensor_functions (19)
sympy.functions.special (17)
sympy.functions (527)
sympy.functions.combinatorial.factorials (1502)
sympy.functions.combinatorial (140)
sympy.functions.combinatorial.numbers (2670)
sympy.utilities.memoization (134)
sympy.functions.elementary.exponential (1387)
sympy.functions.elementary (126)
sympy.functions.elementary.complexes (1836)
sympy.functions.elementary.miscellaneous (1021)
sympy.functions.elementary.piecewise (539)
sympy.functions.elementary.trigonometric (2967)
sympy.functions.elementary.integers (1217)
sympy.functions.elementary.hyperbolic (2420)
sympy.functions.special.error_functions (3150)
sympy.functions.special (129)
sympy.functions.special.hyper (2357)
sympy.functions.special.gamma_functions (1333)
sympy.functions.special.zeta_functions (1035)
sympy.functions.special.tensor_functions (628)
sympy.functions.special.singularity_functions (347)
sympy.functions.special.delta_functions (656)
sympy.polys.polyroots (314)
sympy.polys.polyquinticconst (383)
sympy.functions.special.bsplines (194)
sympy.sets.sets (28)
sympy.sets (281)
sympy.sets.sets (2364)
sympy.sets.fancysets (1342)
sympy.sets.contains (292)
sympy.sets.conditionset (353)
sympy.sets.ordinals (484)
sympy.sets.powerset (293)
sympy.sets.handlers.comparison (2148)
sympy.sets.handlers (129)
sympy.functions.special.bessel (3029)
sympy.polys.orthopolys (162)
sympy.functions.special.polynomials (1730)
sympy.functions.special.spherical_harmonics (565)
sympy.functions.special.elliptic_integrals (814)
sympy.functions.special.beta_functions (594)
sympy.functions.special.mathieu_functions (804)
sympy.tensor.index_methods (191)
sympy.tensor.functions (268)
sympy.tensor.array (216)
sympy.tensor.array.dense_ndim_array (380)
sympy.tensor.array.mutable_ndim_array (128)
sympy.tensor.array.ndim_array (504)
sympy.printing.defaults (128)
sympy.tensor.array.sparse_ndim_array (291)
sympy.tensor.array.arrayop (208)
sympy.tensor.array.array_comprehension (360)
sympy.printing.mathml (654)
sympy.printing.python (195)
sympy.printing.repr (270)
sympy.printing.pycode (559)
sympy.printing.codeprinter (369)
sympy.printing.glsl (243)
sympy.printing.rcode (219)
sympy.printing.jscode (203)
sympy.printing.julia (338)
sympy.printing.mathematica (310)
sympy.printing.octave (340)
sympy.printing.rust (246)
sympy.printing.gtk (143)
sympy.utilities.mathml (143)
sympy.utilities.pkgdata (104)
sympy.printing.preview (183)
sympy.printing.tree (113)
sympy.printing.tableform (162)
sympy.printing.dot (149)
sympy.printing.maple (340)
sympy.utilities.magic (132)
sympy.polys.numberfields (196)
sympy.polys.numberfields.minpoly (382)
sympy.polys.ring_series (285)
sympy.polys.rootoftools (2254)
sympy.polys.numberfields.subfield (195)
sympy.polys.numberfields.utilities (214)
sympy.polys.matrices.exceptions (19)
sympy.polys.matrices (124)
sympy.polys.matrices.domainmatrix (499)
sympy.polys.matrices.exceptions (255)
sympy.polys.matrices.ddm (278)
sympy.polys.matrices.dense (143)
sympy.polys.matrices.sdm (323)
sympy.polys.matrices.domainscalar (159)
sympy.printing.lambdarepr (281)
sympy.printing.numpy (422)
sympy.polys.numberfields.basis (189)
sympy.polys.numberfields.modules (1408)
sympy.polys.matrices.normalforms (170)
sympy.ntheory.modular (144)
sympy.polys.numberfields.exceptions (157)
sympy.polys.numberfields.primes (237)
sympy.polys.partfrac (198)
sympy.matrices (250)
sympy.matrices.common (920)
sympy.matrices.utilities (161)
sympy.matrices.dense (341)
sympy.matrices.decompositions (193)
sympy.matrices.determinant (188)
sympy.matrices.matrices (1276)
sympy.matrices.reductions (146)
sympy.matrices.subspaces (125)
sympy.matrices.eigen (288)
sympy.polys.matrices.eigen (261)
sympy.polys.agca.extensions (224)
sympy.polys.agca (127)
sympy.polys.agca.homomorphisms (279)
sympy.polys.agca.modules (547)
sympy.polys.agca.ideals (247)
sympy.matrices.graph (135)
sympy.matrices.solvers (187)
sympy.matrices.inverse (182)
sympy.matrices.repmatrix (361)
sympy.matrices.sparse (282)
sympy.matrices.sparsetools (147)
sympy.matrices.immutable (1326)
sympy.matrices.expressions (337)
sympy.matrices.expressions.slice (322)
sympy.matrices.expressions.matexpr (2042)
sympy.matrices.expressions.matmul (519)
sympy.strategies (201)
sympy.strategies.rl (253)
sympy.strategies.util (111)
sympy.strategies.traverse (136)
sympy.strategies.core (128)
sympy.strategies.tools (119)
sympy.strategies.branch (150)
sympy.strategies.branch.traverse (157)
sympy.strategies.branch.core (120)
sympy.strategies.branch.tools (104)
sympy.matrices.expressions.inverse (284)
sympy.matrices.expressions.matpow (309)
sympy.matrices.expressions.special (793)
sympy.matrices.expressions.transpose (879)
sympy.matrices.expressions.permutation (417)
sympy.matrices.expressions.matadd (393)
sympy.matrices.expressions.determinant (331)
sympy.matrices.expressions.blockmatrix (10979)
sympy.matrices.expressions.trace (277)
sympy.matrices.expressions.companion (346)
sympy.matrices.expressions.funcmatrix (289)
sympy.matrices.expressions.adjoint (272)
sympy.matrices.expressions.hadamard (511)
sympy.matrices.expressions.diagonal (513)
sympy.matrices.expressions.dotproduct (238)
sympy.matrices.expressions.kronecker (355)
sympy.matrices.expressions.sets (306)
sympy.polys.fields (339)
sympy.series (292)
sympy.series.order (374)
sympy.series.limits (337)
sympy.calculus.accumulationbounds (20)
sympy.calculus (180)
sympy.calculus.euler (164)
sympy.calculus.singularities (143)
sympy.calculus.finite_diff (152)
sympy.calculus.util (266)
sympy.calculus.accumulationbounds (864)
sympy.series.gruntz (294)
sympy.series.series (119)
sympy.series.approximants (136)
sympy.series.residues (125)
sympy.series.sequences (1201)
sympy.series.fourier (436)
sympy.series.series_class (239)
sympy.series.formal (967)
sympy.discrete.convolutions (19)
sympy.discrete (159)
sympy.discrete.transforms (263)
sympy.discrete.convolutions (141)
sympy.series.limitseq (175)
sympy.concrete (136)
sympy.concrete.products (377)
sympy.concrete.expr_with_intlimits (263)
sympy.concrete.expr_with_limits (383)
sympy.concrete.summations (645)
sympy.concrete.gosper (144)
sympy.integrals.integrals (17)
sympy.integrals (190)
sympy.integrals.integrals (557)
sympy.integrals.rationaltools (165)
sympy.integrals.deltafunctions (134)
sympy.integrals.meijerint (586)
sympy.integrals.trigonometry (191)
sympy.integrals.transforms (2649)
sympy.polys.matrices.linsolve (179)
sympy.polys.solvers (355)
sympy.polys.domainmatrix (107)
sympy.integrals.singularityfunctions (147)
sympy.simplify (223)
sympy.simplify.simplify (515)
sympy.simplify.combsimp (156)
sympy.simplify.gammasimp (445)
sympy.simplify.cse_opts (132)
sympy.simplify.hyperexpand (938)
sympy.simplify.powsimp (216)
sympy.simplify.radsimp (278)
sympy.simplify.sqrtdenest (194)
sympy.simplify.trigsimp (332)
sympy.simplify.cse_main (328)
sympy.strategies.tree (139)
sympy.simplify.fu (343)
sympy.simplify.epathtools (165)
sympy.simplify.ratsimp (157)
sympy.solvers (218)
sympy.solvers.solvers (667)
sympy.solvers.polysys (180)
sympy.solvers.bivariate (186)
sympy.solvers.diophantine (140)
sympy.solvers.diophantine.diophantine (1417)
sympy.solvers.solveset (617)
sympy.solvers.recurr (278)
sympy.solvers.ode (160)
sympy.solvers.ode.ode (562)
sympy.solvers.deutils (146)
sympy.solvers.ode.single (988)
sympy.solvers.ode.riccati (206)
sympy.solvers.ode.hypergeometric (156)
sympy.solvers.ode.nonhomogeneous (267)
sympy.solvers.ode.subscheck (159)
sympy.solvers.ode.lie_group (250)
sympy.solvers.pde (255)
sympy.solvers.ode.systems (420)
sympy.solvers.inequalities (307)
sympy.solvers.decompogen (134)
sympy.geometry (203)
sympy.geometry.point (620)
sympy.geometry.exceptions (120)
sympy.geometry.entity (1691)
sympy.sets.handlers.intersection (5639)
sympy.sets.handlers.union (2701)
sympy.geometry.line (1547)
sympy.geometry.util (189)
sympy.geometry.plane (317)
sympy.geometry.ellipse (530)
sympy.geometry.polygon (783)
sympy.geometry.curve (288)
sympy.geometry.parabola (316)
sympy.parsing (168)
sympy.parsing.sympy_parser (753)
ast (435)
_ast (889)
sympy.algebras (155)
sympy.algebras.quaternion (465)
sympy.plotting (181)
sympy.plotting.plot (656)
sympy.plotting.experimental_lambdify (331)
sympy.plotting.textplot (145)
sympy.plotting.plot_implicit (221)
sympy.plotting.intervalmath (150)
sympy.plotting.intervalmath.interval_arithmetic (212)
sympy.plotting.intervalmath.interval_membership (129)
sympy.plotting.intervalmath.lib_interval (173)
sympy.plotting.pygletplot (133)
sympy.interactive (170)
sympy.interactive.printing (304)
sympy.interactive.session (186)
sympy.interactive.traversal (135)
Some relevant discussion is also in #23615
Release Notes
- core
- Import performance of sympy import by lazy loading of testing code
:white_check_mark:
Hi, I am the SymPy bot (v167). I'm here to help you write a release notes entry. Please read the guide on how to write release notes.
Your release notes are in good order.
Here is what the release notes will look like:
- core
- Lazy loading of testing code is used to reduce the time taken for
import sympy. (#23832 by @eendebakpt)
- Lazy loading of testing code is used to reduce the time taken for
This will be added to https://github.com/sympy/sympy/wiki/Release-Notes-for-1.12.
Click here to see the pull request description that was parsed.
<!-- Your title above should be a short description of what
was changed. Do not include the issue number in the title. -->
#### Brief description of what is fixed or changed
The import waterfall of sympy shows 594 modules imported (command `importtime-waterfall sympy`). The total import time (tested with `/bin/test_import`) is
```
Number of tests: 50
The speed of "import sympy" is: 0.272500 +- 0.003218
```
Performance of the sympy import can be improved if the number of imported modules is reduced.
This PR reduced the import time by
* Using a lazy import of `sympy.testing.runtests` in `sympy.utilities.decorator`.
* Creating lazy wrappers for `sympy.testing.test` and `sympy.testing.doctest` in the main `sympy` module.
The lazy loading eliminates the import of `sympy.testing.test`, which triggers loading of `unittest` and `asyncio`.
The new import time is:
```
Number of tests: 50
The speed of "import sympy" is: 0.243919 +- 0.003072
```
The docstrings of `test` and `doctest` are now quite sparse. We could copy the docstrings of the corresponding methods in `sympy.testing.runtests`, or we could deprecate the methods in the main sympy module.
#### Other comments
This PR was written to be backwards compatible. To further reduce the import time, one could consider to not load certain modules by default. For example `sympy.printing.mathematica` could be changed to only load when a user explictly calls `import sympy.printing.mathematica`
Details of the benchmarks:
<details><summary>[MASTER] Result of python bin/test_import</summary>
```
Note: the first run (warm up) was not included in the average + std dev
All runs (including warm up):
[0.2796212559997002, 0.2781042780002281, 0.27615861999993285, 0.2748804050002036, 0.2746562840002298, 0.2732063799999196, 0.27364490100035255, 0.2708097899999302, 0.2752684740003133, 0.270890703999612, 0.2738728619997346, 0.26944935499977873, 0.2704215500002647, 0.27218362399980833, 0.2721846950003055, 0.2744498849997399, 0.26823755899977186, 0.27238844600015, 0.2831268329996419, 0.2742866089997733, 0.27588116900005843, 0.281494990000283, 0.27313322700001663, 0.268633022999893, 0.2680984220000937, 0.27662726300013674, 0.2734363519998624, 0.275800481999795, 0.27083820200004993, 0.2759205409997776, 0.2724356540002191, 0.2689656769998692, 0.272148050999931, 0.2699153190001198, 0.26858375200026785, 0.27064090600015334, 0.2707692060002955, 0.2712384749997909, 0.2706845650000105, 0.2712465599997813, 0.27598768999996537, 0.26832291200025793, 0.2695224099998086, 0.2709240030003457, 0.26965993800013166, 0.27197772999988956, 0.2692769809996207, 0.2698648979999234, 0.2708809119999387, 0.2736668140000802, 0.27018944299970826]
Number of tests: 50
The speed of "import sympy" is: 0.272500 +- 0.003218
```
</details>
<details><summary>[PR] Result of python bin/test_import</summary>
```
Note: the first run (warm up) was not included in the average + std dev
All runs (including warm up):
[0.24775494799996522, 0.24446511799988002, 0.24558605499987607, 0.24339464699960445, 0.24083171199981734, 0.24163932600004046, 0.24593805700033045, 0.25555351100001644, 0.2405642789999547, 0.24108691999981602, 0.2436481409999942, 0.24590180000041073, 0.24222797199990964, 0.2418284290001793, 0.24650318899966805, 0.2435498390000248, 0.24492248100023062, 0.24199547499983964, 0.24216606300024068, 0.24247888899981263, 0.24273960599975908, 0.24888989000010042, 0.2411293659997682, 0.26128544099992723, 0.24476161800021146, 0.24317696499974772, 0.25251023200007694, 0.24237984000001234, 0.24224764900009177, 0.2431937559999824, 0.2482696600000054, 0.24661963899961847, 0.24343341199983115, 0.24264238399973692, 0.24271199299982982, 0.24021964699977616, 0.24562413000012384, 0.24461949899978208, 0.2423081650003951, 0.240376594999816, 0.242632059999778, 0.24419342999999571, 0.24679355099988243, 0.242847428999994, 0.24469787399993947, 0.24236903000019083, 0.2395998730003157, 0.24556445400003213, 0.24200651399996786, 0.24416150100023515, 0.24275182900009895]
Number of tests: 50
The speed of "import sympy" is: 0.244221 +- 0.003791
```
</details>
<details><summary>Result of importtime-waterfall sympy</summary>
```
sympy (1043)
mpmath (1501)
mpmath.usertools (161)
mpmath.ctx_fp (306)
mpmath.ctx_base (383)
mpmath.libmp.backend (29)
mpmath.libmp (302)
mpmath.libmp.libmpf (718)
math (208)
bisect (146)
_bisect (32)
mpmath.libmp.backend (197)
gmpy2 (116)
gmpy (120)
mpmath.libmp.libintmath (489)
mpmath.libmp.libmpc (336)
mpmath.libmp.libelefun (288)
mpmath.libmp.libhyper (274)
mpmath.libmp.gammazeta (844)
mpmath.libmp.libmpi (241)
mpmath.functions.functions (16)
mpmath.functions (440)
mpmath.functions.functions (336)
cmath (52)
mpmath.functions.factorials (169)
mpmath.functions.hypergeometric (269)
mpmath.functions.expintegrals (168)
mpmath.functions.bessel (299)
mpmath.functions.orthogonal (174)
mpmath.functions.theta (145)
mpmath.functions.elliptic (195)
mpmath.functions.zeta (320)
__future__ (156)
mpmath.functions.rszeta (195)
mpmath.functions.zetazeros (261)
mpmath.functions.qfunctions (134)
mpmath.calculus.quadrature (347)
mpmath.calculus (252)
mpmath.calculus.calculus (151)
mpmath.calculus.approximation (134)
mpmath.calculus.differentiation (169)
mpmath.calculus.extrapolation (284)
mpmath.calculus.polynomials (135)
mpmath.calculus.inverselaplace (218)
mpmath.calculus.optimization (298)
copy (205)
weakref (387)
_weakrefset (175)
copyreg (164)
org.python.core (23)
org.python (25)
org (118)
mpmath.calculus.odes (163)
mpmath.matrices.matrices (248)
mpmath.matrices (174)
mpmath.matrices.eigen (194)
mpmath.matrices.eigen_symmetric (187)
mpmath.matrices.calculus (150)
mpmath.matrices.linalg (184)
mpmath.identification (274)
mpmath.visualization (201)
colorsys (128)
mpmath.math2 (241)
mpmath.function_docs (402)
mpmath.ctx_mp (1505)
re (496)
enum (598)
sre_compile (285)
_sre (71)
sre_parse (408)
sre_constants (257)
mpmath.rational (216)
numbers (422)
mpmath.ctx_mp_python (2099)
mpmath.ctx_iv (357)
sympy.release (210)
sympy.core (304)
sympy.core.sympify (332)
typing (1328)
collections.abc (150)
inspect (1344)
dis (388)
opcode (305)
_opcode (144)
linecache (163)
tokenize (797)
token (155)
string (691)
_string (38)
sympy.core.random (244)
sympy.utilities.iterables (20)
sympy.utilities (191)
sympy.utilities.iterables (462)
sympy.utilities.enumerative (182)
sympy.utilities.misc (223)
struct (188)
_struct (61)
textwrap (1060)
sympy.utilities.decorator (219)
sympy.testing.runtests (30)
sympy.testing (135)
sympy.testing.runtests (929)
platform (2244)
traceback (252)
pdb (497)
cmd (196)
bdb (356)
fnmatch (141)
code (216)
codeop (157)
glob (302)
pprint (252)
signal (769)
timeit (231)
gc (47)
doctest (1866)
difflib (551)
unittest (261)
unittest.result (250)
unittest.util (230)
unittest.async_case (192)
asyncio (273)
asyncio.base_events (793)
concurrent.futures (504)
concurrent (127)
concurrent.futures._base (462)
logging (1788)
threading (431)
atexit (38)
socket (1472)
_socket (145)
selectors (535)
select (56)
errno (71)
subprocess (429)
msvcrt (179)
_posixsubprocess (50)
ssl (2423)
_ssl (1403)
base64 (333)
binascii (65)
asyncio.constants (211)
asyncio.coroutines (228)
asyncio.base_futures (143)
asyncio.format_helpers (121)
asyncio.log (111)
asyncio.events (986)
contextvars (128)
_contextvars (119)
asyncio.exceptions (187)
_asyncio (245)
asyncio.base_tasks (138)
asyncio.futures (214)
asyncio.protocols (178)
asyncio.sslproto (329)
asyncio.transports (330)
asyncio.staggered (347)
asyncio.locks (259)
asyncio.tasks (295)
asyncio.trsock (238)
asyncio.runners (143)
asyncio.queues (213)
asyncio.streams (281)
asyncio.subprocess (209)
asyncio.unix_events (674)
asyncio.base_subprocess (234)
asyncio.selector_events (351)
unittest.case (685)
unittest.suite (283)
unittest.loader (565)
unittest.main (226)
argparse (701)
gettext (894)
locale (621)
unittest.runner (273)
unittest.signals (135)
random (397)
_sha512 (46)
_random (46)
shutil (541)
zlib (163)
bz2 (267)
_compression (190)
_bz2 (236)
lzma (255)
_lzma (236)
pwd (61)
grp (51)
tempfile (387)
sympy.core.cache (184)
sympy.external (151)
sympy.external.importtools (315)
sympy.external.gmpy (261)
gmpy2 (130)
sympy.external.pythonmpq (211)
decimal (141)
_decimal (679)
fractions (763)
sympy.utilities.exceptions (188)
sympy.utilities.lambdify (385)
sympy.utilities.source (133)
sympy.utilities.timeutils (129)
sympy.core.parameters (158)
sympy.core.basic (764)
sympy.core.assumptions (28696)
sympy.core.facts (239)
sympy.core.logic (850)
sympy.core.core (171)
sympy.core.sorting (220)
sympy.core.kind (272)
sympy.multipledispatch.dispatcher (19)
sympy.multipledispatch (113)
sympy.multipledispatch.core (175)
sympy.multipledispatch.dispatcher (249)
sympy.multipledispatch.conflict (158)
sympy.multipledispatch.utils (105)
sympy.core._print_helpers (132)
sympy.core.singleton (173)
sympy.core.traversal (180)
sympy.core.expr (1325)
sympy.core.evalf (477)
sympy.core.decorators (156)
sympy.core.mul (644)
sympy.core.operations (505)
sympy.core.numbers (4737)
sympy.core.containers (435)
sympy.core.power (572)
sympy.core.function (1519)
sympy.core.add (525)
sympy.core.rules (135)
sympy.core.symbol (895)
sympy.logic.boolalg (19)
sympy.logic (165)
sympy.logic.boolalg (2942)
sympy.logic.inference (235)
sympy.core.relational (1636)
sympy.core.mod (333)
sympy.core.exprtools (409)
sympy.core.coreerrors (135)
sympy.core.multidimensional (175)
sympy.assumptions (156)
sympy.assumptions.assume (525)
sympy.assumptions.ask (549)
sympy.assumptions.cnf (250)
sympy.assumptions.ask_generated (147)
sympy.assumptions.refine (156)
sympy.assumptions.relation (150)
sympy.assumptions.relation.binrel (397)
sympy.polys (324)
sympy.polys.polytools (1382)
sympy.polys.polyoptions (1415)
sympy.polys.polyerrors (426)
sympy.polys.constructor (178)
sympy.polys.domains (410)
sympy.polys.domains.domain (362)
sympy.polys.domains.domainelement (119)
sympy.polys.orderings (237)
sympy.polys.polyutils (325)
sympy.polys.domains.finitefield (226)
sympy.polys.domains.field (226)
sympy.polys.domains.ring (142)
sympy.polys.domains.modularinteger (192)
sympy.polys.domains.simpledomain (139)
sympy.polys.domains.groundtypes (172)
sympy.polys.domains.pythonrational (106)
sympy.polys.domains.integerring (244)
sympy.polys.domains.characteristiczero (136)
sympy.polys.domains.rationalfield (185)
sympy.polys.domains.algebraicfield (250)
sympy.polys.polyclasses (727)
sympy.polys.densebasic (247)
sympy.polys.monomials (275)
sympy.polys.densearith (242)
sympy.polys.densetools (214)
sympy.polys.euclidtools (288)
sympy.polys.galoistools (293)
sympy.polys.polyconfig (157)
sympy.polys.sqfreetools (168)
sympy.polys.factortools (752)
sympy.polys.rootisolation (413)
sympy.polys.domains.gaussiandomains (343)
sympy.polys.domains.realfield (265)
sympy.polys.domains.mpelements (208)
sympy.polys.domains.complexfield (302)
sympy.polys.domains.polynomialring (224)
sympy.polys.domains.compositedomain (124)
sympy.polys.domains.fractionfield (187)
sympy.polys.domains.expressiondomain (240)
sympy.polys.domains.expressionrawdomain (166)
sympy.polys.domains.pythonfinitefield (166)
sympy.polys.domains.pythonintegerring (149)
sympy.polys.domains.gmpyfinitefield (164)
sympy.polys.domains.gmpyintegerring (149)
sympy.polys.domains.pythonrationalfield (219)
sympy.polys.domains.gmpyrationalfield (170)
sympy.polys.fglmtools (167)
sympy.polys.groebnertools (279)
sympy.polys.rationaltools (128)
sympy.polys.polyfuncs (207)
sympy.polys.specialpolys (231)
sympy.ntheory (269)
sympy.ntheory.generate (474)
array (60)
sympy.ntheory.primetest (162)
sympy.ntheory.factor_ (1402)
sympy.ntheory.digits (126)
sympy.ntheory.ecm (201)
sympy.ntheory.partitions_ (235)
sympy.ntheory.residue_ntheory (384)
sympy.ntheory.multinomial (127)
sympy.ntheory.continued_fraction (143)
sympy.ntheory.egyptian_fraction (133)
sympy.ntheory.qs (296)
sympy.polys.rings (595)
sympy.polys.compatibility (772)
sympy.polys.heuristicgcd (140)
sympy.printing.defaults (20)
sympy.printing (430)
sympy.printing.pretty (208)
sympy.printing.pretty.pretty (902)
sympy.printing.conventions (287)
sympy.printing.precedence (199)
sympy.printing.printer (239)
sympy.printing.str (572)
sympy.printing.pretty.stringpict (248)
sympy.printing.pretty.pretty_symbology (596)
unicodedata (55)
sympy.core.alphabets (139)
sympy.printing.latex (1778)
sympy.tensor.array (22)
sympy.tensor (199)
sympy.tensor.indexed (744)
sympy.functions.special.tensor_functions (19)
sympy.functions.special (17)
sympy.functions (527)
sympy.functions.combinatorial.factorials (1502)
sympy.functions.combinatorial (140)
sympy.functions.combinatorial.numbers (2670)
sympy.utilities.memoization (134)
sympy.functions.elementary.exponential (1387)
sympy.functions.elementary (126)
sympy.functions.elementary.complexes (1836)
sympy.functions.elementary.miscellaneous (1021)
sympy.functions.elementary.piecewise (539)
sympy.functions.elementary.trigonometric (2967)
sympy.functions.elementary.integers (1217)
sympy.functions.elementary.hyperbolic (2420)
sympy.functions.special.error_functions (3150)
sympy.functions.special (129)
sympy.functions.special.hyper (2357)
sympy.functions.special.gamma_functions (1333)
sympy.functions.special.zeta_functions (1035)
sympy.functions.special.tensor_functions (628)
sympy.functions.special.singularity_functions (347)
sympy.functions.special.delta_functions (656)
sympy.polys.polyroots (314)
sympy.polys.polyquinticconst (383)
sympy.functions.special.bsplines (194)
sympy.sets.sets (28)
sympy.sets (281)
sympy.sets.sets (2364)
sympy.sets.fancysets (1342)
sympy.sets.contains (292)
sympy.sets.conditionset (353)
sympy.sets.ordinals (484)
sympy.sets.powerset (293)
sympy.sets.handlers.comparison (2148)
sympy.sets.handlers (129)
sympy.functions.special.bessel (3029)
sympy.polys.orthopolys (162)
sympy.functions.special.polynomials (1730)
sympy.functions.special.spherical_harmonics (565)
sympy.functions.special.elliptic_integrals (814)
sympy.functions.special.beta_functions (594)
sympy.functions.special.mathieu_functions (804)
sympy.tensor.index_methods (191)
sympy.tensor.functions (268)
sympy.tensor.array (216)
sympy.tensor.array.dense_ndim_array (380)
sympy.tensor.array.mutable_ndim_array (128)
sympy.tensor.array.ndim_array (504)
sympy.printing.defaults (128)
sympy.tensor.array.sparse_ndim_array (291)
sympy.tensor.array.arrayop (208)
sympy.tensor.array.array_comprehension (360)
sympy.printing.mathml (654)
sympy.printing.python (195)
sympy.printing.repr (270)
sympy.printing.pycode (559)
sympy.printing.codeprinter (369)
sympy.printing.glsl (243)
sympy.printing.rcode (219)
sympy.printing.jscode (203)
sympy.printing.julia (338)
sympy.printing.mathematica (310)
sympy.printing.octave (340)
sympy.printing.rust (246)
sympy.printing.gtk (143)
sympy.utilities.mathml (143)
sympy.utilities.pkgdata (104)
sympy.printing.preview (183)
sympy.printing.tree (113)
sympy.printing.tableform (162)
sympy.printing.dot (149)
sympy.printing.maple (340)
sympy.utilities.magic (132)
sympy.polys.numberfields (196)
sympy.polys.numberfields.minpoly (382)
sympy.polys.ring_series (285)
sympy.polys.rootoftools (2254)
sympy.polys.numberfields.subfield (195)
sympy.polys.numberfields.utilities (214)
sympy.polys.matrices.exceptions (19)
sympy.polys.matrices (124)
sympy.polys.matrices.domainmatrix (499)
sympy.polys.matrices.exceptions (255)
sympy.polys.matrices.ddm (278)
sympy.polys.matrices.dense (143)
sympy.polys.matrices.sdm (323)
sympy.polys.matrices.domainscalar (159)
sympy.printing.lambdarepr (281)
sympy.printing.numpy (422)
sympy.polys.numberfields.basis (189)
sympy.polys.numberfields.modules (1408)
sympy.polys.matrices.normalforms (170)
sympy.ntheory.modular (144)
sympy.polys.numberfields.exceptions (157)
sympy.polys.numberfields.primes (237)
sympy.polys.partfrac (198)
sympy.matrices (250)
sympy.matrices.common (920)
sympy.matrices.utilities (161)
sympy.matrices.dense (341)
sympy.matrices.decompositions (193)
sympy.matrices.determinant (188)
sympy.matrices.matrices (1276)
sympy.matrices.reductions (146)
sympy.matrices.subspaces (125)
sympy.matrices.eigen (288)
sympy.polys.matrices.eigen (261)
sympy.polys.agca.extensions (224)
sympy.polys.agca (127)
sympy.polys.agca.homomorphisms (279)
sympy.polys.agca.modules (547)
sympy.polys.agca.ideals (247)
sympy.matrices.graph (135)
sympy.matrices.solvers (187)
sympy.matrices.inverse (182)
sympy.matrices.repmatrix (361)
sympy.matrices.sparse (282)
sympy.matrices.sparsetools (147)
sympy.matrices.immutable (1326)
sympy.matrices.expressions (337)
sympy.matrices.expressions.slice (322)
sympy.matrices.expressions.matexpr (2042)
sympy.matrices.expressions.matmul (519)
sympy.strategies (201)
sympy.strategies.rl (253)
sympy.strategies.util (111)
sympy.strategies.traverse (136)
sympy.strategies.core (128)
sympy.strategies.tools (119)
sympy.strategies.branch (150)
sympy.strategies.branch.traverse (157)
sympy.strategies.branch.core (120)
sympy.strategies.branch.tools (104)
sympy.matrices.expressions.inverse (284)
sympy.matrices.expressions.matpow (309)
sympy.matrices.expressions.special (793)
sympy.matrices.expressions.transpose (879)
sympy.matrices.expressions.permutation (417)
sympy.matrices.expressions.matadd (393)
sympy.matrices.expressions.determinant (331)
sympy.matrices.expressions.blockmatrix (10979)
sympy.matrices.expressions.trace (277)
sympy.matrices.expressions.companion (346)
sympy.matrices.expressions.funcmatrix (289)
sympy.matrices.expressions.adjoint (272)
sympy.matrices.expressions.hadamard (511)
sympy.matrices.expressions.diagonal (513)
sympy.matrices.expressions.dotproduct (238)
sympy.matrices.expressions.kronecker (355)
sympy.matrices.expressions.sets (306)
sympy.polys.fields (339)
sympy.series (292)
sympy.series.order (374)
sympy.series.limits (337)
sympy.calculus.accumulationbounds (20)
sympy.calculus (180)
sympy.calculus.euler (164)
sympy.calculus.singularities (143)
sympy.calculus.finite_diff (152)
sympy.calculus.util (266)
sympy.calculus.accumulationbounds (864)
sympy.series.gruntz (294)
sympy.series.series (119)
sympy.series.approximants (136)
sympy.series.residues (125)
sympy.series.sequences (1201)
sympy.series.fourier (436)
sympy.series.series_class (239)
sympy.series.formal (967)
sympy.discrete.convolutions (19)
sympy.discrete (159)
sympy.discrete.transforms (263)
sympy.discrete.convolutions (141)
sympy.series.limitseq (175)
sympy.concrete (136)
sympy.concrete.products (377)
sympy.concrete.expr_with_intlimits (263)
sympy.concrete.expr_with_limits (383)
sympy.concrete.summations (645)
sympy.concrete.gosper (144)
sympy.integrals.integrals (17)
sympy.integrals (190)
sympy.integrals.integrals (557)
sympy.integrals.rationaltools (165)
sympy.integrals.deltafunctions (134)
sympy.integrals.meijerint (586)
sympy.integrals.trigonometry (191)
sympy.integrals.transforms (2649)
sympy.polys.matrices.linsolve (179)
sympy.polys.solvers (355)
sympy.polys.domainmatrix (107)
sympy.integrals.singularityfunctions (147)
sympy.simplify (223)
sympy.simplify.simplify (515)
sympy.simplify.combsimp (156)
sympy.simplify.gammasimp (445)
sympy.simplify.cse_opts (132)
sympy.simplify.hyperexpand (938)
sympy.simplify.powsimp (216)
sympy.simplify.radsimp (278)
sympy.simplify.sqrtdenest (194)
sympy.simplify.trigsimp (332)
sympy.simplify.cse_main (328)
sympy.strategies.tree (139)
sympy.simplify.fu (343)
sympy.simplify.epathtools (165)
sympy.simplify.ratsimp (157)
sympy.solvers (218)
sympy.solvers.solvers (667)
sympy.solvers.polysys (180)
sympy.solvers.bivariate (186)
sympy.solvers.diophantine (140)
sympy.solvers.diophantine.diophantine (1417)
sympy.solvers.solveset (617)
sympy.solvers.recurr (278)
sympy.solvers.ode (160)
sympy.solvers.ode.ode (562)
sympy.solvers.deutils (146)
sympy.solvers.ode.single (988)
sympy.solvers.ode.riccati (206)
sympy.solvers.ode.hypergeometric (156)
sympy.solvers.ode.nonhomogeneous (267)
sympy.solvers.ode.subscheck (159)
sympy.solvers.ode.lie_group (250)
sympy.solvers.pde (255)
sympy.solvers.ode.systems (420)
sympy.solvers.inequalities (307)
sympy.solvers.decompogen (134)
sympy.geometry (203)
sympy.geometry.point (620)
sympy.geometry.exceptions (120)
sympy.geometry.entity (1691)
sympy.sets.handlers.intersection (5639)
sympy.sets.handlers.union (2701)
sympy.geometry.line (1547)
sympy.geometry.util (189)
sympy.geometry.plane (317)
sympy.geometry.ellipse (530)
sympy.geometry.polygon (783)
sympy.geometry.curve (288)
sympy.geometry.parabola (316)
sympy.parsing (168)
sympy.parsing.sympy_parser (753)
ast (435)
_ast (889)
sympy.algebras (155)
sympy.algebras.quaternion (465)
sympy.plotting (181)
sympy.plotting.plot (656)
sympy.plotting.experimental_lambdify (331)
sympy.plotting.textplot (145)
sympy.plotting.plot_implicit (221)
sympy.plotting.intervalmath (150)
sympy.plotting.intervalmath.interval_arithmetic (212)
sympy.plotting.intervalmath.interval_membership (129)
sympy.plotting.intervalmath.lib_interval (173)
sympy.plotting.pygletplot (133)
sympy.interactive (170)
sympy.interactive.printing (304)
sympy.interactive.session (186)
sympy.interactive.traversal (135)
```
</details>
Some relevant discussion is also in #23615
#### Release Notes
<!-- Write the release notes for this release below between the BEGIN and END
statements. The basic format is a bulleted list with the name of the subpackage
and the release note for this PR. For example:
* solvers
* Added a new solver for logarithmic equations.
* functions
* Fixed a bug with log of integers.
or if no release note(s) should be included use:
NO ENTRY
See https://github.com/sympy/sympy/wiki/Writing-Release-Notes for more
information on how to write release notes. The bot will check your release
notes automatically to see if they are formatted correctly. -->
<!-- BEGIN RELEASE NOTES -->
* core
* Lazy loading of testing code is used to reduce the time taken for `import sympy`.
<!-- END RELEASE NOTES -->
Update
The release notes on the wiki have been updated.
Benchmark results from GitHub Actions
Lower numbers are good, higher numbers are bad. A ratio less than 1
means a speed up and greater than 1 means a slowdown. Green lines
beginning with + are slowdowns (the PR is slower then master or
master is slower than the previous release). Red lines beginning
with - are speedups.
Significantly changed benchmark results (PR vs master)
Significantly changed benchmark results (master vs previous release)
before after ratio
[26f7bdbe] [5eb59bda]
<sympy-1.11^0>
- 1.19±0.01ms 737±8μs 0.62 solve.TimeSparseSystem.time_linear_eq_to_matrix(10)
- 3.43±0.06ms 1.37±0.02ms 0.40 solve.TimeSparseSystem.time_linear_eq_to_matrix(20)
- 6.87±0.02ms 2.03±0.04ms 0.30 solve.TimeSparseSystem.time_linear_eq_to_matrix(30)
Full benchmark results can be found as artifacts in GitHub Actions (click on checks at the top of the PR).
![github-actions[bot] avatar](https://avatars.githubusercontent.com/in/15368?v=4 "Published by a pub.dev verified publisher"){kind=small}
This is a good idea.
The docstrings of
testanddoctestare now quite sparse.
Maybe there could be a special class/function for wrapping another function lazily. Something like this:
def lazy_function(module, name):
from importlib import import_module
func = None
def _get_function():
nonlocal func
if func is None:
func = getattr(import_module(module), name)
return func
# The metaclass is needed so that help() shows the docstring
class LazyFunctionMeta(type):
@property
def __doc__(self):
return _get_function().__doc__
class LazyFunction(metaclass=LazyFunctionMeta):
def __call__(self, *args, **kwargs):
return _get_function()(*args, **kwargs)
@property
def __doc__(self):
return _get_function().__doc__
return LazyFunction()
With that you can do test = lazy_function('sympy.testing.runtests', 'test'). The docstring should work and the function is callable.
This PR was written to be backwards compatible. To further reduce the import time, one could consider to not load certain modules by default.
This is something that would have to be weighed very carefully.
For example
sympy.printing.mathematicacould be changed to only load when a user explictly callsimport sympy.printing.mathematica
Why specifically is mathematica a problem?
@oscarbenjamin I added the lazy_function to the PR (with a __str__ and __repr__ added). We could pass also __annotations__, or even pass any attribute, but I have no strong preference here. The location is sympy.core.cache, but perhaps there is a better location. I briefly searched pypi to see wether there are any external packages we could use, but the ones around seem to have a slightly different purpose.
Import times have not changed.
To be clear: there is nothing wrong with mathematica. I just expect that sympy.printing.mathematica is not used in most of the sympy sessions (just as sympy.printing.octave and several others). For functions that are i) not performance critical ii) are not expected to be used by most users, we could use the lazy_function. E.g. replace
from .julia import julia_code
from .mathematica import mathematica_code
with
julia_code=lazy_function('sympy.printing.julia', 'julia_code')
mathematica_code=lazy_function('sympy.printing.mathematica', 'mathematica_code')
This should save about 1 ms import time. In another PR we could go through all candidates for such a lazy import and see whether we can gain a significant import time reduction.
The location is
sympy.core.cache, but perhaps there is a better location.
Another possibility could be sympy/external/importtools.py. I'm not sure what makes the most sense.
In another PR we could go through all candidates for such a lazy import and see whether we can gain a significant import time reduction.
That sounds like a good idea.
Why are unittest and asyncio imported? Neither of those are explicitly referenced anywhere in the codebase.
By the way, some lazy importing functionality is being proposed to be added to core Python itself https://peps.python.org/pep-0690/. Presumably that PEP would make this whole discussion unnecessary if implemented.
I'm all for making imports in general work more lazily, assuming we can do it in a way that minimizes code complexity. I believe some other large libraries do some similar things so it might be worth looking at what else is out there.
Why are
unittestandasyncioimported? Neither of those are explicitly referenced anywhere in the codebase.
sympy.utilities.decorator imports sympy.testing.runtests, which imports doctest, which imports unittest, which imports asyncio. This PR makes the sympy.testing.runtests loading lazy, and thereby avoids imports of the other modules.
There is the https://github.com/mnmelo/lazy_import which has a lazy_callable that provides functionality similar to the lazy_function in this PR. The package has received no new commits since 2018, so I decided not to use this external package.
For performance I inlined the function lookup in __call__. For a very simple function the lazy_function overhead goes down from
237 ns ± 5.31 ns per loop (mean ± std. dev. of 7 runs, 1000000 loops each)
to
200 ns ± 3.95 ns per loop (mean ± std. dev. of 7 runs, 1000000 loops each)
I think it would be better to test this in test_submodule_imports.py rather than add a new bin/test_* script.
Both test failures seem to be unrelated.
@oscarbenjamin I rebased, but there are still two tests failing. It seems related to a recent change in aesara where aesara.tensor.abs_ is deprecated:
https://github.com/aesara-devs/aesara/blame/9f176da71d41635e8854fd601fd6a68102b0c6e5/aesara/tensor/math.py#L3122
Also related is https://github.com/sympy/sympy/issues/23733, but that did not resolve the issue.
I left a comment here: https://github.com/aesara-devs/aesara/pull/1054#issuecomment-1222172591