Removing distutils for 3.12 compatibility
Distutils has been deprecated in 3.10 and removed in 3.12 (https://docs.python.org/3.10/whatsnew/3.10.html#distutils-deprecated)
For 3.12 compatibility (#2129) I'm proposing to remove distutils.
distutils was only used for its StrictVersion class, this can be replaced by packaging's Version class in every instance, as this library is already a dependency this requires no further packages to be added as dependency.
For the use of the Version class, see: https://packaging.pypa.io/en/latest/version.html
Thanks @Cyanosite for this pull request. The change looks good to me.
CI: https://gitlab.com/coremltools1/coremltools/-/pipelines/1288882002
error: subprocess-exited-with-error
× Getting requirements to build wheel did not run successfully.
│ exit code: 1
╰─> [33 lines of output]
Traceback (most recent call last):
File "/Users/cyanosite/Development/coremltools/envs/coremltools-py3.12/lib/python3.12/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 353, in <module>
main()
File "/Users/cyanosite/Development/coremltools/envs/coremltools-py3.12/lib/python3.12/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 335, in main
json_out['return_val'] = hook(**hook_input['kwargs'])
^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/cyanosite/Development/coremltools/envs/coremltools-py3.12/lib/python3.12/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 112, in get_requires_for_build_wheel
backend = _build_backend()
^^^^^^^^^^^^^^^^
File "/Users/cyanosite/Development/coremltools/envs/coremltools-py3.12/lib/python3.12/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 77, in _build_backend
obj = import_module(mod_path)
^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/cyanosite/Development/coremltools/envs/coremltools-py3.12/lib/python3.12/importlib/__init__.py", line 90, in import_module
return _bootstrap._gcd_import(name[level:], package, level)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "<frozen importlib._bootstrap>", line 1387, in _gcd_import
File "<frozen importlib._bootstrap>", line 1360, in _find_and_load
File "<frozen importlib._bootstrap>", line 1310, in _find_and_load_unlocked
File "<frozen importlib._bootstrap>", line 488, in _call_with_frames_removed
File "<frozen importlib._bootstrap>", line 1387, in _gcd_import
File "<frozen importlib._bootstrap>", line 1360, in _find_and_load
File "<frozen importlib._bootstrap>", line 1331, in _find_and_load_unlocked
File "<frozen importlib._bootstrap>", line 935, in _load_unlocked
File "<frozen importlib._bootstrap_external>", line 995, in exec_module
File "<frozen importlib._bootstrap>", line 488, in _call_with_frames_removed
File "/private/var/folders/tn/j2ynrhds2zd4fcp7sv3jxwym0000gn/T/pip-build-env-1u1blzlr/overlay/lib/python3.12/site-packages/setuptools/__init__.py", line 16, in <module>
import setuptools.version
File "/private/var/folders/tn/j2ynrhds2zd4fcp7sv3jxwym0000gn/T/pip-build-env-1u1blzlr/overlay/lib/python3.12/site-packages/setuptools/version.py", line 1, in <module>
import pkg_resources
File "/private/var/folders/tn/j2ynrhds2zd4fcp7sv3jxwym0000gn/T/pip-build-env-1u1blzlr/overlay/lib/python3.12/site-packages/pkg_resources/__init__.py", line 2172, in <module>
register_finder(pkgutil.ImpImporter, find_on_path)
^^^^^^^^^^^^^^^^^^^
AttributeError: module 'pkgutil' has no attribute 'ImpImporter'. Did you mean: 'zipimporter'?
[end of output]
note: This error originates from a subprocess, and is likely not a problem with pip.
error: subprocess-exited-with-error
× Getting requirements to build wheel did not run successfully.
│ exit code: 1
╰─> See above for output.
note: This error originates from a subprocess, and is likely not a problem with pip.
I'm getting the test pipelines to run locally so you don't have to test my PR every time I submit changes. I ran into the above issue while trying to run the tests. https://stackoverflow.com/questions/77364550/attributeerror-module-pkgutil-has-no-attribute-impimporter-did-you-mean pkgutil.ImpImporter was long deprecated and has been removed in 3.12, due to this I will be trying to do something about this error too. I will get back to you once I have a working solution.
Alternatively I could work on that in a separate PR and just test with 3.10 for now
The test pipeline isn't going to work with Python 3.12. We'll need to update all of our test dependencies to version that support Python 3.12.
I recommend you stick with 3.10 for this pull request.
rootdir: /Users/cyanosite/Development/coremltools, configfile: pytest.ini
plugins: flake8-1.0.7, cov-5.0.0, xdist-2.5.0, sugar-1.0.0, mock-3.8.2, timeout-2.3.1, forked-1.6.0
timeout: 600.0s
timeout method: signal
timeout func_only: False
collected 0 items
================================================================== warnings summary ==================================================================
<frozen importlib._bootstrap>:914
<frozen importlib._bootstrap>:914
<frozen importlib._bootstrap>:914
<frozen importlib._bootstrap>:914
<frozen importlib._bootstrap>:914
<frozen importlib._bootstrap>:914
<frozen importlib._bootstrap>:914
<frozen importlib._bootstrap>:914: ImportWarning: _SixMetaPathImporter.find_spec() not found; falling back to find_module()
<frozen importlib._bootstrap>:671
<frozen importlib._bootstrap>:671: ImportWarning: _SixMetaPathImporter.exec_module() not found; falling back to load_module()
-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
------------------------------- generated xml file: /Users/cyanosite/Development/coremltools/build/py-test-report.xml --------------------------------
Results (2.24s):
ERROR: module or package not found: coremltools.test (missing __init__.py?)
I cannot figure out why the pipeline doesn't work locally, seems to be the same error as in the pipeline on gitlab. Flake reports no errors, that runs fine.
Updated CI: https://gitlab.com/coremltools1/coremltools/-/pipelines/1289337174
I fixed the remaining importerrors reported in the pipeline, should be good now. Managed to get the tests working locally, the test that failed on the pipeline now runs for me with only one error.
> assert(_HAS_KMEANS1D)
E AssertionError
../../envs/coremltools-py3.10/lib/python3.10/site-packages/coremltools/models/neural_network/quantization_utils.py:404: AssertionError
This might be something to do with dependencies on my side rather than the pull request. It might run fine on the pipeline. However the optimizations tests use quite a lot of swap so this might not be something I can run locally often.
Updated CI: https://gitlab.com/coremltools1/coremltools/-/pipelines/1290878130