Mac M2 Installation Error

[firmai]

Processing /Users/dereksnow/Sovai/GitHub/SovAI/notebooks/studies/pyspi Installing build dependencies ... done Getting requirements to build wheel ... done Preparing metadata (pyproject.toml) ... done Collecting scikit-learn==1.0.1 (from pyspi-lib==0.4.2) Using cached scikit-learn-1.0.1.tar.gz (6.6 MB) Installing build dependencies ... done Getting requirements to build wheel ... done Preparing metadata (pyproject.toml) ... error error: subprocess-exited-with-error

× Preparing metadata (pyproject.toml) did not run successfully. │ exit code: 1 ╰─> [1527 lines of output] Partial import of sklearn during the build process. :128: DeprecationWarning:

    `numpy.distutils` is deprecated since NumPy 1.23.0, as a result
    of the deprecation of `distutils` itself. It will be removed for
    Python >= 3.12. For older Python versions it will remain present.
    It is recommended to use `setuptools < 60.0` for those Python versions.
    For more details, see:
      https://numpy.org/devdocs/reference/distutils_status_migration.html
  
  
  INFO: C compiler: clang -Wsign-compare -Wunreachable-code -fno-common -dynamic -DNDEBUG -g -fwrapv -O3 -Wall -isysroot /Library/Developer/CommandLineTools/SDKs/MacOSX14.sdk
  
  INFO: compile options: '-c'
  INFO: clang: test_program.c
  INFO: clang objects/test_program.o -o test_program
  INFO: C compiler: clang -Wsign-compare -Wunreachable-code -fno-common -dynamic -DNDEBUG -g -fwrapv -O3 -Wall -isysroot /Library/Developer/CommandLineTools/SDKs/MacOSX14.sdk
  
  INFO: compile options: '-c'
  extra options: '-fopenmp'
  INFO: clang: test_program.c
  clang: error: unsupported option '-fopenmp'
  /private/var/folders/tj/2qbc2n2x1234_l7b3z5y06740000gn/T/pip-install-se08yce2/scikit-learn_96e7f55684d048fa808c01b4a5ac7177/sklearn/_build_utils/openmp_helpers.py:126: UserWarning:
  
                  ***********
                  * WARNING *
                  ***********
  
  It seems that scikit-learn cannot be built with OpenMP.
  
  - Make sure you have followed the installation instructions:
  
      https://scikit-learn.org/dev/developers/advanced_installation.html
  
  - If your compiler supports OpenMP but you still see this
    message, please submit a bug report at:
  
      https://github.com/scikit-learn/scikit-learn/issues
  
  - The build will continue with OpenMP-based parallelism
    disabled. Note however that some estimators will run in
    sequential mode instead of leveraging thread-based
    parallelism.
  
                      ***
  
    warnings.warn(message)
  Compiling sklearn/__check_build/_check_build.pyx because it changed.
  Compiling sklearn/preprocessing/_csr_polynomial_expansion.pyx because it changed.
  Compiling sklearn/cluster/_dbscan_inner.pyx because it changed.
  Compiling sklearn/cluster/_hierarchical_fast.pyx because it changed.
  Compiling sklearn/cluster/_k_means_common.pyx because it changed.
  Compiling sklearn/cluster/_k_means_lloyd.pyx because it changed.
  Compiling sklearn/cluster/_k_means_elkan.pyx because it changed.
  Compiling sklearn/cluster/_k_means_minibatch.pyx because it changed.
  Compiling sklearn/datasets/_svmlight_format_fast.pyx because it changed.
  Compiling sklearn/decomposition/_online_lda_fast.pyx because it changed.
  Compiling sklearn/decomposition/_cdnmf_fast.pyx because it changed.
  Compiling sklearn/ensemble/_gradient_boosting.pyx because it changed.
  Compiling sklearn/ensemble/_hist_gradient_boosting/_gradient_boosting.pyx because it changed.
  Compiling sklearn/ensemble/_hist_gradient_boosting/histogram.pyx because it changed.
  Compiling sklearn/ensemble/_hist_gradient_boosting/splitting.pyx because it changed.
  Compiling sklearn/ensemble/_hist_gradient_boosting/_binning.pyx because it changed.
  Compiling sklearn/ensemble/_hist_gradient_boosting/_predictor.pyx because it changed.
  Compiling sklearn/ensemble/_hist_gradient_boosting/_loss.pyx because it changed.
  Compiling sklearn/ensemble/_hist_gradient_boosting/_bitset.pyx because it changed.
  Compiling sklearn/ensemble/_hist_gradient_boosting/common.pyx because it changed.
  Compiling sklearn/ensemble/_hist_gradient_boosting/utils.pyx because it changed.
  Compiling sklearn/feature_extraction/_hashing_fast.pyx because it changed.
  Compiling sklearn/manifold/_utils.pyx because it changed.
  Compiling sklearn/manifold/_barnes_hut_tsne.pyx because it changed.
  Compiling sklearn/metrics/cluster/_expected_mutual_info_fast.pyx because it changed.
  Compiling sklearn/metrics/_pairwise_fast.pyx because it changed.
  Compiling sklearn/neighbors/_ball_tree.pyx because it changed.
  Compiling sklearn/neighbors/_kd_tree.pyx because it changed.
  Compiling sklearn/neighbors/_partition_nodes.pyx because it changed.
  Compiling sklearn/neighbors/_dist_metrics.pyx because it changed.
  Compiling sklearn/neighbors/_typedefs.pyx because it changed.
  Compiling sklearn/neighbors/_quad_tree.pyx because it changed.
  Compiling sklearn/tree/_tree.pyx because it changed.
  Compiling sklearn/tree/_splitter.pyx because it changed.
  Compiling sklearn/tree/_criterion.pyx because it changed.
  Compiling sklearn/tree/_utils.pyx because it changed.
  Compiling sklearn/utils/sparsefuncs_fast.pyx because it changed.
  Compiling sklearn/utils/_cython_blas.pyx because it changed.
  Compiling sklearn/utils/arrayfuncs.pyx because it changed.
  Compiling sklearn/utils/murmurhash.pyx because it changed.
  Compiling sklearn/utils/_fast_dict.pyx because it changed.
  Compiling sklearn/utils/_openmp_helpers.pyx because it changed.
  Compiling sklearn/utils/_seq_dataset.pyx because it changed.
  Compiling sklearn/utils/_weight_vector.pyx because it changed.
  Compiling sklearn/utils/_random.pyx because it changed.
  Compiling sklearn/utils/_logistic_sigmoid.pyx because it changed.
  Compiling sklearn/utils/_readonly_array_wrapper.pyx because it changed.
  Compiling sklearn/svm/_newrand.pyx because it changed.
  Compiling sklearn/svm/_libsvm.pyx because it changed.
  Compiling sklearn/svm/_liblinear.pyx because it changed.
  Compiling sklearn/svm/_libsvm_sparse.pyx because it changed.
  Compiling sklearn/linear_model/_cd_fast.pyx because it changed.
  Compiling sklearn/linear_model/_sgd_fast.pyx because it changed.
  Compiling sklearn/linear_model/_sag_fast.pyx because it changed.
  Compiling sklearn/_isotonic.pyx because it changed.
  warning: sklearn/cluster/_dbscan_inner.pyx:17:5: Only extern functions can throw C++ exceptions.
  warning: sklearn/neighbors/_dist_metrics.pxd:19:64: The keyword 'nogil' should appear at the end of the function signature line. Placing it before 'except' or 'noexcept' will be disallowed in a future version of Cython.
  warning: sklearn/neighbors/_dist_metrics.pxd:29:65: The keyword 'nogil' should appear at the end of the function signature line. Placing it before 'except' or 'noexcept' will be disallowed in a future version of Cython.
  warning: sklearn/neighbors/_dist_metrics.pxd:38:79: The keyword 'nogil' should appear at the end of the function signature line. Placing it before 'except' or 'noexcept' will be disallowed in a future version of Cython.
  warning: sklearn/neighbors/_dist_metrics.pxd:42:79: The keyword 'nogil' should appear at the end of the function signature line. Placing it before 'except' or 'noexcept' will be disallowed in a future version of Cython.
  warning: sklearn/neighbors/_dist_metrics.pxd:61:51: The keyword 'nogil' should appear at the end of the function signature line. Placing it before 'except' or 'noexcept' will be disallowed in a future version of Cython.
  warning: sklearn/neighbors/_dist_metrics.pxd:64:52: The keyword 'nogil' should appear at the end of the function signature line. Placing it before 'except' or 'noexcept' will be disallowed in a future version of Cython.
  warning: sklearn/neighbors/_dist_metrics.pxd:71:68: The keyword 'nogil' should appear at the end of the function signature line. Placing it before 'except' or 'noexcept' will be disallowed in a future version of Cython.
  warning: sklearn/neighbors/_dist_metrics.pxd:73:67: The keyword 'nogil' should appear at the end of the function signature line. Placing it before 'except' or 'noexcept' will be disallowed in a future version of Cython.
  performance hint: sklearn/cluster/_k_means_common.pyx:31:5: Exception check on '_euclidean_dense_dense' will always require the GIL to be acquired. Declare '_euclidean_dense_dense' as 'noexcept' if you control the definition and you're sure you don't want the function to raise exceptions.
  performance hint: sklearn/cluster/_k_means_common.pyx:63:5: Exception check on '_euclidean_sparse_dense' will always require the GIL to be acquired. Declare '_euclidean_sparse_dense' as 'noexcept' if you control the definition and you're sure you don't want the function to raise exceptions.
  performance hint: sklearn/cluster/_k_means_common.pyx:120:40: Exception check after calling '__pyx_fuse_0_euclidean_dense_dense' will always require the GIL to be acquired. Declare '__pyx_fuse_0_euclidean_dense_dense' as 'noexcept' if you control the definition and you're sure you don't want the function to raise exceptions.
  performance hint: sklearn/cluster/_k_means_common.pyx:120:40: Exception check after calling '__pyx_fuse_1_euclidean_dense_dense' will always require the GIL to be acquired. Declare '__pyx_fuse_1_euclidean_dense_dense' as 'noexcept' if you control the definition and you're sure you don't want the function to raise exceptions.
  performance hint: sklearn/cluster/_k_means_common.pyx:154:41: Exception check after calling '__pyx_fuse_0_euclidean_sparse_dense' will always require the GIL to be acquired. Declare '__pyx_fuse_0_euclidean_sparse_dense' as 'noexcept' if you control the definition and you're sure you don't want the function to raise exceptions.
  performance hint: sklearn/cluster/_k_means_common.pyx:154:41: Exception check after calling '__pyx_fuse_1_euclidean_sparse_dense' will always require the GIL to be acquired. Declare '__pyx_fuse_1_euclidean_sparse_dense' as 'noexcept' if you control the definition and you're sure you don't want the function to raise exceptions.
  performance hint: sklearn/cluster/_k_means_elkan.pyx:336:5: Exception check on '_update_chunk_dense' will always require the GIL to be acquired.
  Possible solutions:
      1. Declare '_update_chunk_dense' as 'noexcept' if you control the definition and you're sure you don't want the function to raise exceptions.
      2. Use an 'int' return type on '_update_chunk_dense' to allow an error code to be returned.
  performance hint: sklearn/cluster/_k_means_elkan.pyx:571:5: Exception check on '_update_chunk_sparse' will always require the GIL to be acquired.
  Possible solutions:
      1. Declare '_update_chunk_sparse' as 'noexcept' if you control the definition and you're sure you don't want the function to raise exceptions.
      2. Use an 'int' return type on '_update_chunk_sparse' to allow an error code to be returned.
  performance hint: sklearn/cluster/_k_means_elkan.pyx:88:41: Exception check after calling '__pyx_fuse_0_euclidean_dense_dense' will always require the GIL to be acquired. Declare '__pyx_fuse_0_euclidean_dense_dense' as 'noexcept' if you control the definition and you're sure you don't want the function to raise exceptions.
  performance hint: sklearn/cluster/_k_means_elkan.pyx:93:45: Exception check after calling '__pyx_fuse_0_euclidean_dense_dense' will always require the GIL to be acquired. Declare '__pyx_fuse_0_euclidean_dense_dense' as 'noexcept' if you control the definition and you're sure you don't want the function to raise exceptions.
  performance hint: sklearn/cluster/_k_means_elkan.pyx:88:41: Exception check after calling '__pyx_fuse_1_euclidean_dense_dense' will always require the GIL to be acquired. Declare '__pyx_fuse_1_euclidean_dense_dense' as 'noexcept' if you control the definition and you're sure you don't want the function to raise exceptions.
  performance hint: sklearn/cluster/_k_means_elkan.pyx:93:45: Exception check after calling '__pyx_fuse_1_euclidean_dense_dense' will always require the GIL to be acquired. Declare '__pyx_fuse_1_euclidean_dense_dense' as 'noexcept' if you control the definition and you're sure you don't want the function to raise exceptions.
  performance hint: sklearn/cluster/_k_means_elkan.pyx:164:42: Exception check after calling '__pyx_fuse_0_euclidean_sparse_dense' will always require the GIL to be acquired. Declare '__pyx_fuse_0_euclidean_sparse_dense' as 'noexcept' if you control the definition and you're sure you don't want the function to raise exceptions.
  performance hint: sklearn/cluster/_k_means_elkan.pyx:172:46: Exception check after calling '__pyx_fuse_0_euclidean_sparse_dense' will always require the GIL to be acquired. Declare '__pyx_fuse_0_euclidean_sparse_dense' as 'noexcept' if you control the definition and you're sure you don't want the function to raise exceptions.
  performance hint: sklearn/cluster/_k_means_elkan.pyx:164:42: Exception check after calling '__pyx_fuse_1_euclidean_sparse_dense' will always require the GIL to be acquired. Declare '__pyx_fuse_1_euclidean_sparse_dense' as 'noexcept' if you control the definition and you're sure you don't want the function to raise exceptions.
  performance hint: sklearn/cluster/_k_means_elkan.pyx:172:46: Exception check after calling '__pyx_fuse_1_euclidean_sparse_dense' will always require the GIL to be acquired. Declare '__pyx_fuse_1_euclidean_sparse_dense' as 'noexcept' if you control the definition and you're sure you don't want the function to raise exceptions.
  performance hint: sklearn/cluster/_k_means_elkan.pyx:294:31: Exception check after calling '__pyx_fuse_0_update_chunk_dense' will always require the GIL to be acquired.
  Possible solutions:
      1. Declare '__pyx_fuse_0_update_chunk_dense' as 'noexcept' if you control the definition and you're sure you don't want the function to raise exceptions.
      2. Use an 'int' return type on '__pyx_fuse_0_update_chunk_dense' to allow an error code to be returned.
  performance hint: sklearn/cluster/_k_means_elkan.pyx:294:31: Exception check after calling '__pyx_fuse_1_update_chunk_dense' will always require the GIL to be acquired.
  Possible solutions:

[firmai]

I think it could be because I am using Python 11 and above. Do you think you would at some point work on the compatibility? Thanks again for the nice package.

[jmoo2880]

Hi @firmai, Yes that is correct, pyspi currently supports python versions 3.8 and 3.9. This is because pyspi has several dependencies that clash in later versions of python (>=3.10).

[firmai]

I would love to work with you to get a working version for 3.11. Do you prefer to wait for the dependencies to catch up, or write some conditional code:

import sys

if sys.version_info < (3, 10):
    # Code for Python versions less than 3.10
else:
    # Code for Python 3.10 and above

Python 3.9 will probably be at its end of life next year.

[jmoo2880]

@firmai

I'd be happy to work on bringing pyspi to Python 3.10+. A few challenges worth considering. The way that testing of SPIs is currently setup is to "benchmark" each on a synthetic dataset from a simulated model (coupled map lattice). After any major changes to the codebase or dependency updates, we re-compute the outputs of the SPIs on the same dataset and compare to the reference values (a snapshot taken at pyspi v0.4.2). If the differences exceed some pre-determined (SPI-specific) threshold, those SPIs are flagged for investigation.

Without a theoretical ground truth for (some) SPIs, inspecting differences in the outputs of the SPIs when bumping dependencies doesn't discriminate between genuine improvements in how the SPI is calculated and whether the differences are a result of some algorithmic/syntax modification (e.g., changes in how a function is called or how the output is returned). In the latter case, the function may still produce an output without breaking the code (so it passes the unit tests), but the values themselves may be ultimately meaningless. So the process of bumping dependencies such that pyspi is compatible with python 3.10+ needs to be approached with some caution, particularly if we want results to be reproducible across pyspi versions.

[jmoo2880]

Pyspi v1.1.0 brings support to Python 3.10, 3.11 and 3.12. See https://github.com/DynamicsAndNeuralSystems/pyspi/pull/70.