Add support for python 3.12+ and drop support for 3.7

[instinct-vfx]

Fixes #1948. Builds on #1943 as many of the issues are due to outdated vendored dependencies.

[codecov[bot]]

Codecov Report

:x: Patch coverage is 75.60976% with 10 lines in your changes missing coverage. Please review. :white_check_mark: Project coverage is 59.98%. Comparing base (96cd33c) to head (86331da). :warning: Report is 4 commits behind head on main.

Files with missing lines	Patch %	Lines
src/rez/utils/pip.py	84.84%	2 Missing and 3 partials :warning:
src/rez/utils/py_dist.py	0.00%	4 Missing :warning:
src/rez/bind/rez.py	0.00%	1 Missing :warning:

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #1950      +/-   ##
==========================================
- Coverage   60.07%   59.98%   -0.10%     
==========================================
  Files         163      163              
  Lines       20185    20118      -67     
  Branches     3510     3519       +9     
==========================================
- Hits        12126    12067      -59     
+ Misses       7241     7230      -11     
- Partials      818      821       +3     

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.

:rocket: New features to boost your workflow:

• :snowflake: Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[JeanChristopheMorinPerso]

Tests are now passing. We should still try to install stuff with rez-pip, just in case. Our test suite doesn't have any tests for that.

[JeanChristopheMorinPerso]

Also, this PR technically drops support for Python 3.7 since we use importlib.metadata.

[JeanChristopheMorinPerso]

@instinct-vfx @maxnbk @bpabel do you have anything against merging this PR? If not, can you all give your approval please?

[Aphosis]

Hi everyone,

Just a heads up, I stumbled upon this PR while debugging an issue with rez pip, and while it solves some issues it looks like it also introduces new ones.

For some context, I tried to install keyring with the following command (using the latest rez tag at first):

rez pip --install --python-version=3.10  keyring==25.2.1

It failed with the following traceback:

Traceback (most recent call last):
  File "<frozen runpy>", line 198, in _run_module_as_main
  File "<frozen runpy>", line 88, in _run_code
  File "C:\Users\user\rez-install\Scripts\rez\rez.exe\__main__.py", line 7, in <module>
  File "C:\Users\user\projects\rez\src\rez\cli\_entry_points.py", line 69, in run_rez
    return run()
           ^^^^^
  File "C:\Users\user\projects\rez\src\rez\cli\_main.py", line 189, in run
    returncode = run_cmd()
                 ^^^^^^^^^
  File "C:\Users\user\projects\rez\src\rez\cli\_main.py", line 181, in run_cmd
    return func(opts, opts.parser, extra_arg_groups)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\user\projects\rez\src\rez\cli\pip.py", line 52, in command
    pip_install_package(
  File "C:\Users\user\projects\rez\src\rez\pip.py", line 344, in pip_install_package
    rez_requires = get_rez_requirements(
                   ^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\user\projects\rez\src\rez\utils\pip.py", line 405, in get_rez_requirements
    is_pure_python = is_pure_python_package(installed_dist)
                     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\user\projects\rez\src\rez\utils\pip.py", line 329, in is_pure_python_package
    wheel_data = setuptools_dist.get_metadata('WHEEL')
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
AttributeError: 'NoneType' object has no attribute 'get_metadata'

After some debugging it looked like it came from pkg_resources and distlib assigning slightly different names to the discovered distributions.

In particular, it was failing on jaraco.functools, since pkg_resources discovered it as jaraco_functools.

I manually tested discovering installed distributions with importlib.metadata and found out that it correctly gave jaraco.functools, so before writing a patch I checked to see if someone already opened an issue on GitHub and that's when I found your PR.

I went ahead and tested it with the same command, and it gave me another error:

Traceback (most recent call last):
  File "<frozen runpy>", line 198, in _run_module_as_main
  File "<frozen runpy>", line 88, in _run_code
  File "C:\Users\user\rez-install\Scripts\rez\rez.exe\__main__.py", line 7, in <module>
  File "C:\Users\user\projects\rez\src\rez\cli\_entry_points.py", line 69, in run_rez
    return run()
           ^^^^^
  File "C:\Users\user\projects\rez\src\rez\cli\_main.py", line 189, in run
    returncode = run_cmd()
                 ^^^^^^^^^
  File "C:\Users\user\projects\rez\src\rez\cli\_main.py", line 181, in run_cmd
    return func(opts, opts.parser, extra_arg_groups)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\user\projects\rez\src\rez\cli\pip.py", line 52, in command
    pip_install_package(
  File "C:\Users\user\projects\rez\src\rez\pip.py", line 344, in pip_install_package
    rez_requires = get_rez_requirements(
                   ^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\user\projects\rez\src\rez\utils\pip.py", line 420, in get_rez_requirements
    is_pure_python = is_pure_python_package(installed_dist)
                     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\user\projects\rez\src\rez\utils\pip.py", line 332, in is_pure_python_package
    for f in setuptools_dist.files:
             ^^^^^^^^^^^^^^^^^^^^^
AttributeError: 'NoneType' object has no attribute 'files'

After debugging it, it turns out this time it was failing on importlib_metadata: both importlib and distlib correctly gave the name importlib_metadata, but since the distlib name is further normalized by rez using (the now localized in rez) pkg_resources.safe_name function, it actual comparison in rez.utils.pip.convert_distlib_to_setuptools becomes "importlib_metadata" == "importlib-metadata", thus failing to map the distribution from importlib to distlib.

I also tried it with flask, sphinx, and requests just to see if that was not an isolated case, and only requests (and its dependencies) was installed successfully.

I can open a separate issue if needed, but since that looked like a regression I opted to comment here first.

If you need anything else I'll be happy to answer with more information!

[JeanChristopheMorinPerso]

Thank you @Aphosis for the report! I can reproduce locally. I'll try to find a fix.

[JeanChristopheMorinPerso]

Umm, for what it's worth, it also fails with rez 3.2.1 on python 3.11.

[JeanChristopheMorinPerso]

@Aphosis I pushed a fix. Can you test it and let us know if it fixes your problem please? Please take a close look at the generated package names. If you see any difference in the rez packages created (capital letters, etc), please let us know.

[Aphosis]

@Aphosis I pushed a fix. Can you test it and let us know if it fixes your problem please? Please take a close look at the generated package names. If you see any difference in the rez packages created (capital letters, etc), please let us know.

@JeanChristopheMorinPerso With the new commit I was able to install keyring without any issue.

I also tested with a few different packages (requests, sphinx, flask) and did not notice anything wrong.

Thanks a lot for your quick answer and fix!