Fixing version collection

[aguschin]

I kept running with issues with the automatic 3rd party dependencies (inferring the wrong version, or missing packages). The version issue is due to some python packages having a mismatch between MODULE.__version__ and the version required for pip:

• After installing regex==2023.6.3, import regex ; regex.__version__ gives 2.5.129 . But there is no 2.5.129 version of regex so the install fail
• After installing opencv-python-headless=4.8.0.74, import cv2 ; cv2.__version__ gives 4.8.0 but there is no 4.8.0 version of opencv-python-headless (you need to give the 4th-level version).

Reported in discord.

[omesser]

@aguschin have you tried to see if importlib.metadata ? They cover packages and distributions. I believe this should work as of python 3.10 (before that we may need to patch or copy packages_distributions() helper but that's it:

>>> packages = importlib.metadata.packages_distributions()
>>> packages
{'pip': ['pip'], 'cv2': ['opencv-python-headless'], '_distutils_hack': ['setuptools'], 'pkg_resources': ['setuptools'], 'setuptools': ['setuptools'], 'regex': ['regex'], 'wheel': ['wheel'], 'numpy': ['numpy']}
>>> importlib.metadata.distribution(packages['cv2'][0]).version
'4.8.0.74'

works for regex as well:

>>> importlib.metadata.distribution(packages['regex'][0]).version
'2023.6.3'

[omesser]

@aguschin issued a patch in https://github.com/iterative/mlem/pull/694