import `monty.json` slowing down core import

[DanielYang59]

Summary

• from monty.json import xxx significantly slowly down core modules, as torch is eagerly imported
• [ ] Might need to look into other commonly used modules

For example:

python -X importtime -c "from pymatgen.core.structure import Structure" 2> pmg.log && tuna pmg.log

With torch installed (required by optional matgl):

After removing torch:

[janosh]

this is probably import overhead from loading pandas and maybe other big packages. if you start an interactive python session first, wait for it to load and then run poscar = Structure.from_file("./POSCAR") on its own, i expect it will be faster

[DanielYang59]

Thanks for the input! I'm not quite sure as I didn't have time to do a thorough investigation (and I don't have extensive experience of import time profiling either). I would look into this later (input from people with more knowledge on this would be hugely appreciated of course).

I think for DFT calculations, input file import should usually not be the bottleneck, but almost two second to read such a simple file sound too much for me.

[janosh]

have a look at https://github.com/materialsproject/pymatgen/issues/3563 which tried to reduce startup time by lazy-importing pandas only in code that actually needs it. that seems to have back-fired though based on this comment https://github.com/materialsproject/pymatgen/issues/3563#issuecomment-1907031585. this takes more careful testing than i did in https://github.com/materialsproject/pymatgen/pull/3568 and https://github.com/materialsvirtuallab/monty/pull/604 to make sure any changes really improve startup time

[DanielYang59]

This should also be partially fixed by https://github.com/materialsvirtuallab/monty/pull/713, after that I would dig deeper and see what else is slowing down import