Missing modules (libcoremlpython) when importing coremltools 8.2 on linux

[bbcevoli]

🐞Describing the bug

Coremltools 8.2 installs fine on Linux but errors with missing modules when imported. Looks like libcoremlpython.so is missing.

❯ python3 -c "import coremltools"
Failed to load _MLModelProxy: No module named 'coremltools.libcoremlpython'
Failed to load _MLCPUComputeDeviceProxy: No module named 'coremltools.libcoremlpython'
Failed to load _MLGPUComputeDeviceProxy: No module named 'coremltools.libcoremlpython'
Failed to load _MLNeuralEngineComputeDeviceProxy: No module named 'coremltools.libcoremlpython'
Failed to load _MLModelProxy: No module named 'coremltools.libcoremlpython'
Failed to load _MLComputePlanProxy: No module named 'coremltools.libcoremlpython'
Failed to load _MLModelProxy: No module named 'coremltools.libcoremlpython'
Failed to load _MLModelAssetProxy: No module named 'coremltools.libcoremlpython'
❯ find ./myenv/lib/python3.12/site-packages/coremltools -name "*.so"
./myenv/lib/python3.12/site-packages/coremltools/_deps/kmeans1d/_core.cpython-312-x86_64-linux-gnu.so
./myenv/lib/python3.12/site-packages/coremltools/libmilstoragepython.so
./myenv/lib/python3.12/site-packages/coremltools/libmodelpackage.so

I only see this only on Linux. On macOS, I can install and import coremltools 8.2 fine (I can see libcoremlpython.so in /myenv/lib/python3.12/site-packages/coremltools/).

Do you have any advice?

Stack Trace

❯ python3 -c "import coremltools"
Failed to load _MLModelProxy: No module named 'coremltools.libcoremlpython'
Failed to load _MLCPUComputeDeviceProxy: No module named 'coremltools.libcoremlpython'
Failed to load _MLGPUComputeDeviceProxy: No module named 'coremltools.libcoremlpython'
Failed to load _MLNeuralEngineComputeDeviceProxy: No module named 'coremltools.libcoremlpython'
Failed to load _MLModelProxy: No module named 'coremltools.libcoremlpython'
Failed to load _MLComputePlanProxy: No module named 'coremltools.libcoremlpython'
Failed to load _MLModelProxy: No module named 'coremltools.libcoremlpython'
Failed to load _MLModelAssetProxy: No module named 'coremltools.libcoremlpython'

To Reproduce

❯ python3.12 -m venv myenv
❯ source myenv/bin/activate
❯ pip install coremltools
Collecting coremltools
  Downloading coremltools-8.2-cp312-none-manylinux1_x86_64.whl.metadata (2.5 kB)
Requirement already satisfied: numpy>=1.14.5 in ./myenv/lib/python3.12/site-packages (from coremltools) (2.2.3)
Requirement already satisfied: protobuf>=3.1.0 in ./myenv/lib/python3.12/site-packages (from coremltools) (6.30.0)
Requirement already satisfied: sympy in ./myenv/lib/python3.12/site-packages (from coremltools) (1.13.3)
Requirement already satisfied: tqdm in ./myenv/lib/python3.12/site-packages (from coremltools) (4.67.1)
Requirement already satisfied: packaging in ./myenv/lib/python3.12/site-packages (from coremltools) (24.2)
Requirement already satisfied: attrs>=21.3.0 in ./myenv/lib/python3.12/site-packages (from coremltools) (25.1.0)
Requirement already satisfied: cattrs in ./myenv/lib/python3.12/site-packages (from coremltools) (24.1.2)
Requirement already satisfied: pyaml in ./myenv/lib/python3.12/site-packages (from coremltools) (25.1.0)
Requirement already satisfied: PyYAML in ./myenv/lib/python3.12/site-packages (from pyaml->coremltools) (6.0.2)
Requirement already satisfied: mpmath<1.4,>=1.1.0 in ./myenv/lib/python3.12/site-packages (from sympy->coremltools) (1.3.0)
Downloading coremltools-8.2-cp312-none-manylinux1_x86_64.whl (2.2 MB)
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 2.2/2.2 MB 21.7 MB/s eta 0:00:00
Installing collected packages: coremltools
Successfully installed coremltools-8.2

[notice] A new release of pip is available: 24.3.1 -> 25.0.1
[notice] To update, run: pip install --upgrade pip
❯ python3 -c "import coremltools"
Failed to load _MLModelProxy: No module named 'coremltools.libcoremlpython'
Failed to load _MLCPUComputeDeviceProxy: No module named 'coremltools.libcoremlpython'
Failed to load _MLGPUComputeDeviceProxy: No module named 'coremltools.libcoremlpython'
Failed to load _MLNeuralEngineComputeDeviceProxy: No module named 'coremltools.libcoremlpython'
Failed to load _MLModelProxy: No module named 'coremltools.libcoremlpython'
Failed to load _MLComputePlanProxy: No module named 'coremltools.libcoremlpython'
Failed to load _MLModelProxy: No module named 'coremltools.libcoremlpython'
Failed to load _MLModelAssetProxy: No module named 'coremltools.libcoremlpython'
❯ find ./myenv/lib/python3.12/site-packages/coremltools -name "*.so"
./myenv/lib/python3.12/site-packages/coremltools/_deps/kmeans1d/_core.cpython-312-x86_64-linux-gnu.so
./myenv/lib/python3.12/site-packages/coremltools/libmilstoragepython.so
./myenv/lib/python3.12/site-packages/coremltools/libmodelpackage.so

System environment (please complete the following information):

• coremltools version: 8.2
• OS (e.g. MacOS version or Linux type): Linux jammy (22.04)
• Any other relevant version information (e.g. PyTorch or TensorFlow version): na

[TobyRoseman]

libcoremlpython.so not being included in our Linux wheels is intentional. That shared object file provides Python access to the Core ML Framework. This Framework is not available on Linux.

Looking at our codebase, it seems all these message are just warnings. So you should actually be able to successfully import coremltools on Linux.

We should change it so that we only output these warnings when we are not on Linux.