yt icon indicating copy to clipboard operation
yt copied to clipboard

BLD: Cython 3 compatibility

Open neutrinoceros opened this issue 3 years ago • 6 comments

Bug report

Bug summary

I report that I've attempted to build yt from source with Cython 3.0 (alpha8) and got the following error. I don't think that fixing this urgent (or necessary ?) but it probably should be discussed or addressed at some point.

Traceback
Obtaining file:///Users/robcleme/dev/yt-project/yt-py310
    Preparing wheel metadata ... error
    ERROR: Command errored out with exit status 1:
     command: /Users/robcleme/.pyenv/versions/yt-py310/bin/python /Users/robcleme/.pyenv/versions/yt-py310/lib/python3.10/site-packages/pip/_vendor/pep517/in_process/_in_process.py prepare_metadata_for_build_wheel /var/folders/xh/yfmp7l9n3x50ln2yvrndt7c801pbkm/T/tmpfctihuv8
         cwd: /Users/robcleme/dev/yt-project/yt-py310
    Complete output (281 lines):
    Unable to compile OpenMP test program so Cython
    extensions will be compiled without parallel support
    running dist_info
    creating /private/var/folders/xh/yfmp7l9n3x50ln2yvrndt7c801pbkm/T/pip-modern-metadata-ty2wpdmk/yt.egg-info
    writing /private/var/folders/xh/yfmp7l9n3x50ln2yvrndt7c801pbkm/T/pip-modern-metadata-ty2wpdmk/yt.egg-info/PKG-INFO
    writing dependency_links to /private/var/folders/xh/yfmp7l9n3x50ln2yvrndt7c801pbkm/T/pip-modern-metadata-ty2wpdmk/yt.egg-info/dependency_links.txt
    writing entry points to /private/var/folders/xh/yfmp7l9n3x50ln2yvrndt7c801pbkm/T/pip-modern-metadata-ty2wpdmk/yt.egg-info/entry_points.txt
    writing requirements to /private/var/folders/xh/yfmp7l9n3x50ln2yvrndt7c801pbkm/T/pip-modern-metadata-ty2wpdmk/yt.egg-info/requires.txt
    writing top-level names to /private/var/folders/xh/yfmp7l9n3x50ln2yvrndt7c801pbkm/T/pip-modern-metadata-ty2wpdmk/yt.egg-info/top_level.txt
    writing manifest file '/private/var/folders/xh/yfmp7l9n3x50ln2yvrndt7c801pbkm/T/pip-modern-metadata-ty2wpdmk/yt.egg-info/SOURCES.txt'
    Compiling yt/geometry/fake_octree.pyx because it depends on /Users/robcleme/.pyenv/versions/yt-py310/lib/python3.10/site-packages/Cython/Includes/libc/string.pxd.
    Compiling yt/geometry/grid_container.pyx because it depends on /Users/robcleme/.pyenv/versions/yt-py310/lib/python3.10/site-packages/Cython/Includes/libc/string.pxd.
    Compiling yt/geometry/grid_visitors.pyx because it depends on /Users/robcleme/.pyenv/versions/yt-py310/lib/python3.10/site-packages/Cython/Includes/libc/string.pxd.
    Compiling yt/geometry/oct_container.pyx because it depends on /Users/robcleme/.pyenv/versions/yt-py310/lib/python3.10/site-packages/Cython/Includes/libc/string.pxd.
    Compiling yt/geometry/oct_visitors.pyx because it depends on /Users/robcleme/.pyenv/versions/yt-py310/lib/python3.10/site-packages/Cython/Includes/libc/string.pxd.
    Compiling yt/geometry/particle_deposit.pyx because it depends on /Users/robcleme/.pyenv/versions/yt-py310/lib/python3.10/site-packages/Cython/Includes/numpy/math.pxd.
    Compiling yt/geometry/particle_oct_container.pyx because it depends on /Users/robcleme/.pyenv/versions/yt-py310/lib/python3.10/site-packages/Cython/Includes/numpy/math.pxd.
    Compiling yt/geometry/particle_smooth.pyx because it depends on /Users/robcleme/.pyenv/versions/yt-py310/lib/python3.10/site-packages/Cython/Includes/numpy/math.pxd.
    Compiling yt/geometry/selection_routines.pyx because it depends on /Users/robcleme/.pyenv/versions/yt-py310/lib/python3.10/site-packages/Cython/Includes/libc/string.pxd.
    Compiling yt/utilities/cython_fortran_utils.pyx because it depends on /Users/robcleme/.pyenv/versions/yt-py310/lib/python3.10/site-packages/Cython/Includes/libc/string.pxd.
    Compiling yt/frontends/ramses/io_utils.pyx because it depends on /Users/robcleme/.pyenv/versions/yt-py310/lib/python3.10/site-packages/Cython/Includes/libc/string.pxd.
    Compiling yt/utilities/lib/cykdtree/kdtree.pyx because it depends on /Users/robcleme/.pyenv/versions/yt-py310/lib/python3.10/site-packages/Cython/Includes/libcpp/vector.pxd.
    Compiling yt/utilities/lib/cykdtree/utils.pyx because it depends on /Users/robcleme/.pyenv/versions/yt-py310/lib/python3.10/site-packages/Cython/Includes/libcpp/vector.pxd.
    Compiling yt/frontends/artio/_artio_caller.pyx because it depends on /Users/robcleme/.pyenv/versions/yt-py310/lib/python3.10/site-packages/Cython/Includes/numpy/math.pxd.
    Compiling yt/utilities/lib/_octree_raytracing.pyx because it depends on /Users/robcleme/.pyenv/versions/yt-py310/lib/python3.10/site-packages/Cython/Includes/libcpp/vector.pxd.
    Compiling yt/utilities/lib/allocation_container.pyx because it depends on /Users/robcleme/.pyenv/versions/yt-py310/lib/python3.10/site-packages/Cython/Includes/libc/string.pxd.
    Compiling yt/utilities/lib/alt_ray_tracers.pyx because it depends on /Users/robcleme/.pyenv/versions/yt-py310/lib/python3.10/site-packages/Cython/Includes/libc/string.pxd.
    Compiling yt/utilities/lib/amr_kdtools.pyx because it depends on /Users/robcleme/.pyenv/versions/yt-py310/lib/python3.10/site-packages/Cython/Includes/libc/string.pxd.
    Compiling yt/utilities/lib/autogenerated_element_samplers.pyx because it depends on /Users/robcleme/.pyenv/versions/yt-py310/lib/python3.10/site-packages/Cython/Includes/libc/math.pxd.
    Compiling yt/utilities/lib/basic_octree.pyx because it depends on /Users/robcleme/.pyenv/versions/yt-py310/lib/python3.10/site-packages/Cython/Includes/libc/string.pxd.
    Compiling yt/utilities/lib/bitarray.pyx because it depends on /Users/robcleme/.pyenv/versions/yt-py310/lib/python3.10/site-packages/Cython/Includes/libc/string.pxd.
    Compiling yt/utilities/lib/bounded_priority_queue.pyx because it depends on /Users/robcleme/.pyenv/versions/yt-py310/lib/python3.10/site-packages/Cython/Includes/libc/string.pxd.
    Compiling yt/utilities/lib/bounding_volume_hierarchy.pyx because it depends on /Users/robcleme/.pyenv/versions/yt-py310/lib/python3.10/site-packages/Cython/Includes/libc/string.pxd.
    Compiling yt/utilities/lib/contour_finding.pyx because it depends on /Users/robcleme/.pyenv/versions/yt-py310/lib/python3.10/site-packages/Cython/Includes/libc/string.pxd.
    Compiling yt/utilities/lib/cosmology_time.pyx because it depends on /Users/robcleme/.pyenv/versions/yt-py310/lib/python3.10/site-packages/Cython/Includes/libc/string.pxd.
    Compiling yt/utilities/lib/cyoctree.pyx because it depends on /Users/robcleme/.pyenv/versions/yt-py310/lib/python3.10/site-packages/Cython/Includes/numpy/math.pxd.
    Compiling yt/utilities/lib/depth_first_octree.pyx because it depends on /Users/robcleme/.pyenv/versions/yt-py310/lib/python3.10/site-packages/Cython/Includes/libc/string.pxd.
    Compiling yt/utilities/lib/distance_queue.pyx because it depends on /Users/robcleme/.pyenv/versions/yt-py310/lib/python3.10/site-packages/Cython/Includes/libc/string.pxd.
    Compiling yt/utilities/lib/element_mappings.pyx because it depends on /Users/robcleme/.pyenv/versions/yt-py310/lib/python3.10/site-packages/Cython/Includes/libc/string.pxd.
    Compiling yt/utilities/lib/ewah_bool_wrap.pyx because it depends on /Users/robcleme/.pyenv/versions/yt-py310/lib/python3.10/site-packages/Cython/Includes/libcpp/vector.pxd.
    Compiling yt/utilities/lib/fnv_hash.pyx because it depends on /Users/robcleme/.pyenv/versions/yt-py310/lib/python3.10/site-packages/Cython/Includes/libc/string.pxd.
    Compiling yt/utilities/lib/fortran_reader.pyx because it depends on /Users/robcleme/.pyenv/versions/yt-py310/lib/python3.10/site-packages/Cython/Includes/libc/string.pxd.
    Compiling yt/utilities/lib/geometry_utils.pyx because it depends on /Users/robcleme/.pyenv/versions/yt-py310/lib/python3.10/site-packages/Cython/Includes/libc/string.pxd.
    Compiling yt/utilities/lib/grid_traversal.pyx because it depends on /Users/robcleme/.pyenv/versions/yt-py310/lib/python3.10/site-packages/Cython/Includes/libc/string.pxd.
    Compiling yt/utilities/lib/image_samplers.pyx because it depends on /Users/robcleme/.pyenv/versions/yt-py310/lib/python3.10/site-packages/Cython/Includes/libcpp/vector.pxd.
    Compiling yt/utilities/lib/image_utilities.pyx because it depends on /Users/robcleme/.pyenv/versions/yt-py310/lib/python3.10/site-packages/Cython/Includes/libc/string.pxd.
    Compiling yt/utilities/lib/interpolators.pyx because it depends on /Users/robcleme/.pyenv/versions/yt-py310/lib/python3.10/site-packages/Cython/Includes/libc/string.pxd.
    Compiling yt/utilities/lib/lenses.pyx because it depends on /Users/robcleme/.pyenv/versions/yt-py310/lib/python3.10/site-packages/Cython/Includes/libc/string.pxd.
    Compiling yt/utilities/lib/line_integral_convolution.pyx because it depends on /Users/robcleme/.pyenv/versions/yt-py310/lib/python3.10/site-packages/Cython/Includes/libc/string.pxd.
    Compiling yt/utilities/lib/marching_cubes.pyx because it depends on /Users/robcleme/.pyenv/versions/yt-py310/lib/python3.10/site-packages/Cython/Includes/libc/string.pxd.
    Compiling yt/utilities/lib/mesh_triangulation.pyx because it depends on /Users/robcleme/.pyenv/versions/yt-py310/lib/python3.10/site-packages/Cython/Includes/libc/string.pxd.
    Compiling yt/utilities/lib/mesh_utilities.pyx because it depends on /Users/robcleme/.pyenv/versions/yt-py310/lib/python3.10/site-packages/Cython/Includes/libc/string.pxd.
    Compiling yt/utilities/lib/misc_utilities.pyx because it depends on /Users/robcleme/.pyenv/versions/yt-py310/lib/python3.10/site-packages/Cython/Includes/libc/string.pxd.
    Compiling yt/utilities/lib/origami.pyx because it depends on /Users/robcleme/.pyenv/versions/yt-py310/lib/python3.10/site-packages/Cython/Includes/libc/string.pxd.
    Compiling yt/utilities/lib/particle_kdtree_tools.pyx because it depends on /Users/robcleme/.pyenv/versions/yt-py310/lib/python3.10/site-packages/Cython/Includes/numpy/math.pxd.
    Compiling yt/utilities/lib/particle_mesh_operations.pyx because it depends on /Users/robcleme/.pyenv/versions/yt-py310/lib/python3.10/site-packages/Cython/Includes/libc/string.pxd.
    Compiling yt/utilities/lib/partitioned_grid.pyx because it depends on /Users/robcleme/.pyenv/versions/yt-py310/lib/python3.10/site-packages/Cython/Includes/libc/string.pxd.
    Compiling yt/utilities/lib/pixelization_routines.pyx because it depends on /Users/robcleme/.pyenv/versions/yt-py310/lib/python3.10/site-packages/Cython/Includes/numpy/math.pxd.
    Compiling yt/utilities/lib/points_in_volume.pyx because it depends on /Users/robcleme/.pyenv/versions/yt-py310/lib/python3.10/site-packages/Cython/Includes/libc/string.pxd.
    Compiling yt/utilities/lib/primitives.pyx because it depends on /Users/robcleme/.pyenv/versions/yt-py310/lib/python3.10/site-packages/Cython/Includes/libc/string.pxd.
    Compiling yt/utilities/lib/quad_tree.pyx because it depends on /Users/robcleme/.pyenv/versions/yt-py310/lib/python3.10/site-packages/Cython/Includes/libc/string.pxd.
    Compiling yt/utilities/lib/ragged_arrays.pyx because it depends on /Users/robcleme/.pyenv/versions/yt-py310/lib/python3.10/site-packages/Cython/Includes/libc/string.pxd.
    Compiling yt/utilities/lib/write_array.pyx because it depends on /Users/robcleme/.pyenv/versions/yt-py310/lib/python3.10/site-packages/Cython/Includes/libc/string.pxd.

    Error compiling Cython file:
    ------------------------------------------------------------
    ...
                         fields = None, int domain_id = -1,
                         int domain_offset = 0,
                         periodicity = (True, True, True),
                         index_fields = None,
                         OctreeContainer particle_octree = None,
                         np.int64_t [:] pdom_ind = None,
                                                  ^
    ------------------------------------------------------------

    yt/geometry/particle_smooth.pyx:62:47: Compiler crash in AnalyseExpressionsTransform

    File 'Nodes.py', line 396, in analyse_expressions: StatListNode(particle_smooth.pyx:12:0)
    File 'Nodes.py', line 396, in analyse_expressions: StatListNode(particle_smooth.pyx:34:5)
    File 'Nodes.py', line 5233, in analyse_expressions: CClassDefNode(particle_smooth.pyx:34:5,
        as_name = 'ParticleSmoothOperation',
        class_name = 'ParticleSmoothOperation',
        module_name = '',
        punycode_class_name = 'ParticleSmoothOperation',
        visibility = 'private')
    File 'Nodes.py', line 396, in analyse_expressions: StatListNode(particle_smooth.pyx:35:4)
    File 'Nodes.py', line 340, in analyse_expressions: CompilerDirectivesNode(particle_smooth.pyx:49:4)
    File 'Nodes.py', line 396, in analyse_expressions: StatListNode(particle_smooth.pyx:49:4)
    File 'Nodes.py', line 5707, in analyse_expressions: SingleAssignmentNode(particle_smooth.pyx:49:4)
    File 'Nodes.py', line 5827, in analyse_types: SingleAssignmentNode(particle_smooth.pyx:49:4)
    File 'ExprNodes.py', line 9395, in analyse_types: PyCFunctionNode(particle_smooth.pyx:49:4,
        binding = True,
        is_temp = 1,
        pymethdef_cname = '__pyx_mdef_2yt_8geometry_15particle_smooth_23ParticleSmoothOperation_7process_octree',
        result_is_used = True,
        use_managed_ref = True)
    File 'ExprNodes.py', line 9478, in analyse_default_args: PyCFunctionNode(particle_smooth.pyx:49:4,
        binding = True,
        is_temp = 1,
        pymethdef_cname = '__pyx_mdef_2yt_8geometry_15particle_smooth_23ParticleSmoothOperation_7process_octree',
        result_is_used = True,
        use_managed_ref = True)
    File 'ExprNodes.py', line 8069, in analyse_types: TupleNode(particle_smooth.pyx:49:4,
        is_sequence_constructor = 1,
        result_is_used = True,
        use_managed_ref = True)
    File 'ExprNodes.py', line 719, in analyse_types: CoerceToMemViewSliceNode(particle_smooth.pyx:62:47,
        is_temp = 1,
        result_is_used = True,
        use_managed_ref = True)
    File 'ExprNodes.py', line 486, in not_implemented: CoerceToMemViewSliceNode(particle_smooth.pyx:62:47,
        is_temp = 1,
        result_is_used = True,
        use_managed_ref = True)

    Compiler crash traceback from this point on:
      File "/Users/robcleme/.pyenv/versions/yt-py310/lib/python3.10/site-packages/Cython/Compiler/ExprNodes.py", line 486, in not_implemented
        raise InternalError(
    InternalError: Internal compiler error: CoerceToMemViewSliceNode.analyse_types not implemented
    Traceback (most recent call last):
      File "/Users/robcleme/.pyenv/versions/yt-py310/lib/python3.10/site-packages/Cython/Build/Dependencies.py", line 1307, in cythonize_one_helper
        return cythonize_one(*m)
      File "/Users/robcleme/.pyenv/versions/yt-py310/lib/python3.10/site-packages/Cython/Build/Dependencies.py", line 1283, in cythonize_one
        raise CompileError(None, pyx_file)
    Cython.Compiler.Errors.CompileError: yt/geometry/particle_smooth.pyx
    warning: yt/geometry/particle_oct_container.pyx:847:33: Non-trivial type declarators in shared declaration (e.g. mix of pointers and values). Each pointer declaration should be on its own line.
    warning: yt/geometry/particle_oct_container.pyx:847:46: Non-trivial type declarators in shared declaration (e.g. mix of pointers and values). Each pointer declaration should be on its own line.
    multiprocessing.pool.RemoteTraceback:
    """
    Traceback (most recent call last):
      File "/Users/robcleme/.pyenv/versions/3.10.0b2/lib/python3.10/multiprocessing/pool.py", line 125, in worker
        result = (True, func(*args, **kwds))
      File "/Users/robcleme/.pyenv/versions/3.10.0b2/lib/python3.10/multiprocessing/pool.py", line 48, in mapstar
        return list(map(*args))
      File "/Users/robcleme/.pyenv/versions/yt-py310/lib/python3.10/site-packages/Cython/Build/Dependencies.py", line 1307, in cythonize_one_helper
        return cythonize_one(*m)
      File "/Users/robcleme/.pyenv/versions/yt-py310/lib/python3.10/site-packages/Cython/Build/Dependencies.py", line 1283, in cythonize_one
        raise CompileError(None, pyx_file)
    Cython.Compiler.Errors.CompileError: yt/geometry/particle_smooth.pyx
    """

    The above exception was the direct cause of the following exception:

    Traceback (most recent call last):
      File "/Users/robcleme/.pyenv/versions/yt-py310/lib/python3.10/site-packages/pip/_vendor/pep517/in_process/_in_process.py", line 280, in <module>
    [ 8/53] Cythonizing yt/geometry/grid_container.pyx
    [20/53] Cythonizing yt/utilities/lib/basic_octree.pyx
    [26/53] Cythonizing yt/utilities/lib/cykdtree/kdtree.pyx
    [43/53] Cythonizing yt/utilities/lib/mesh_utilities.pyx
    [ 3/53] Cythonizing yt/utilities/lib/cykdtree/utils.pyx
    [16/53] Cythonizing yt/utilities/cython_fortran_utils.pyx
    [19/53] Cythonizing yt/utilities/lib/amr_kdtools.pyx
    [33/53] Cythonizing yt/utilities/lib/fortran_reader.pyx
    [38/53] Cythonizing yt/utilities/lib/interpolators.pyx
    [42/53] Cythonizing yt/utilities/lib/mesh_triangulation.pyx
    [ 7/53] Cythonizing yt/geometry/fake_octree.pyx
    [17/53] Cythonizing yt/utilities/lib/allocation_container.pyx
    [31/53] Cythonizing yt/utilities/lib/ewah_bool_wrap.pyx
    [ 5/53] Cythonizing yt/frontends/artio/_artio_caller.pyx
    [29/53] Cythonizing yt/utilities/lib/distance_queue.pyx
    [36/53] Cythonizing yt/utilities/lib/image_samplers.pyx
        main()
    [ 1/53] Cythonizing yt/utilities/lib/_octree_raytracing.pyx
    [15/53] Cythonizing yt/geometry/selection_routines.pyx
    [49/53] Cythonizing yt/utilities/lib/points_in_volume.pyx
    [53/53] Cythonizing yt/utilities/lib/write_array.pyx
      File "/Users/robcleme/.pyenv/versions/yt-py310/lib/python3.10/site-packages/pip/_vendor/pep517/in_process/_in_process.py", line 263, in main
    [ 4/53] Cythonizing yt/utilities/lib/misc_utilities.pyx
    [28/53] Cythonizing yt/utilities/lib/depth_first_octree.pyx
    [32/53] Cythonizing yt/utilities/lib/fnv_hash.pyx
    [40/53] Cythonizing yt/utilities/lib/line_integral_convolution.pyx
    [46/53] Cythonizing yt/utilities/lib/particle_mesh_operations.pyx
    [12/53] Cythonizing yt/geometry/particle_deposit.pyx
    [25/53] Cythonizing yt/utilities/lib/cosmology_time.pyx
    [30/53] Cythonizing yt/utilities/lib/element_mappings.pyx
    [35/53] Cythonizing yt/utilities/lib/grid_traversal.pyx
    [51/53] Cythonizing yt/utilities/lib/quad_tree.pyx
    [10/53] Cythonizing yt/geometry/oct_container.pyx
    [24/53] Cythonizing yt/utilities/lib/contour_finding.pyx
    [44/53] Cythonizing yt/utilities/lib/origami.pyx
    [47/53] Cythonizing yt/utilities/lib/partitioned_grid.pyx
    [50/53] Cythonizing yt/utilities/lib/primitives.pyx
    [ 6/53] Cythonizing yt/frontends/ramses/io_utils.pyx
    [18/53] Cythonizing yt/utilities/lib/alt_ray_tracers.pyx
    [22/53] Cythonizing yt/utilities/lib/bounded_priority_queue.pyx
    [34/53] Cythonizing yt/utilities/lib/geometry_utils.pyx
    [11/53] Cythonizing yt/geometry/oct_visitors.pyx
    [27/53] Cythonizing yt/utilities/lib/cyoctree.pyx
    [45/53] Cythonizing yt/utilities/lib/particle_kdtree_tools.pyx
    [ 9/53] Cythonizing yt/geometry/grid_visitors.pyx
    [13/53] Cythonizing yt/geometry/particle_oct_container.pyx
    [39/53] Cythonizing yt/utilities/lib/lenses.pyx
    [52/53] Cythonizing yt/utilities/lib/ragged_arrays.pyx
        json_out['return_val'] = hook(**hook_input['kwargs'])
      File "/Users/robcleme/.pyenv/versions/yt-py310/lib/python3.10/site-packages/pip/_vendor/pep517/in_process/_in_process.py", line 133, in prepare_metadata_for_build_wheel
        return hook(metadata_directory, config_settings)
      File "/Users/robcleme/.pyenv/versions/yt-py310/lib/python3.10/site-packages/setuptools/build_meta.py", line 166, in prepare_metadata_for_build_wheel
    [ 2/53] Cythonizing yt/utilities/lib/autogenerated_element_samplers.pyx
    [14/53] Cythonizing yt/geometry/particle_smooth.pyx
    analyse_types not implemented
    Called from: CoerceToMemViewSliceNode.not_implemented 485
    Called from: CoerceToMemViewSliceNode.analyse_types 719
    Called from: TupleNode.analyse_types 8069
    Called from: PyCFunctionNode.analyse_default_args 9478
    Called from: PyCFunctionNode.analyse_types 9395
    Called from: SingleAssignmentNode.analyse_types 5827
    Called from: SingleAssignmentNode.analyse_expressions 5707
    Called from: <listcomp> 396
    Called from: StatListNode.analyse_expressions 396
    Called from: CompilerDirectivesNode.analyse_expressions 340
    Called from: <listcomp> 396
    Called from: StatListNode.analyse_expressions 396
    Called from: CClassDefNode.analyse_expressions 5233
    Called from: <listcomp> 396
    Called from: StatListNode.analyse_expressions 396
    Called from: <listcomp> 396
    Called from: StatListNode.analyse_expressions 396
    Called from: AnalyseExpressionsTransform.visit_ModuleNode 2351
    Called from: AnalyseExpressionsTransform._visit 180
    Called from: AnalyseExpressionsTransform.__call__ 284
    Called from: AnalyseExpressionsTransform.__call__ 302
    Called from: run 352
    Called from: run_pipeline 372
    Called from: run_pipeline 505
    Called from: compile_single 577
    Called from: cythonize_one 1262
    Called from: cythonize_one_helper 1307
    Called from: mapstar 48
    Called from: worker 125
    Called from: SpawnProcess.run 108
    Called from: SpawnProcess._bootstrap 315
    Called from: SpawnProcess._main 129
    Called from: spawn_main 116
    Called from: <module> 1
    ----------------------------------------------------------------------
    [21/53] Cythonizing yt/utilities/lib/bitarray.pyx
    [23/53] Cythonizing yt/utilities/lib/bounding_volume_hierarchy.pyx
    [37/53] Cythonizing yt/utilities/lib/image_utilities.pyx
    [41/53] Cythonizing yt/utilities/lib/marching_cubes.pyx
    [48/53] Cythonizing yt/utilities/lib/pixelization_routines.pyx
        self.run_setup()
      File "/Users/robcleme/.pyenv/versions/yt-py310/lib/python3.10/site-packages/setuptools/build_meta.py", line 258, in run_setup
        super(_BuildMetaLegacyBackend,
      File "/Users/robcleme/.pyenv/versions/yt-py310/lib/python3.10/site-packages/setuptools/build_meta.py", line 150, in run_setup
        exec(compile(code, __file__, 'exec'), locals())
      File "setup.py", line 89, in <module>
        setup(
      File "/Users/robcleme/.pyenv/versions/yt-py310/lib/python3.10/site-packages/setuptools/__init__.py", line 153, in setup
        return distutils.core.setup(**attrs)
      File "/Users/robcleme/.pyenv/versions/3.10.0b2/lib/python3.10/distutils/core.py", line 148, in setup
        dist.run_commands()
      File "/Users/robcleme/.pyenv/versions/3.10.0b2/lib/python3.10/distutils/dist.py", line 966, in run_commands
        self.run_command(cmd)
      File "/Users/robcleme/.pyenv/versions/3.10.0b2/lib/python3.10/distutils/dist.py", line 985, in run_command
        cmd_obj.run()
      File "/Users/robcleme/.pyenv/versions/yt-py310/lib/python3.10/site-packages/setuptools/command/dist_info.py", line 31, in run
        egg_info.run()
      File "/Users/robcleme/.pyenv/versions/yt-py310/lib/python3.10/site-packages/setuptools/command/egg_info.py", line 299, in run
        self.find_sources()
      File "/Users/robcleme/.pyenv/versions/yt-py310/lib/python3.10/site-packages/setuptools/command/egg_info.py", line 306, in find_sources
        mm.run()
      File "/Users/robcleme/.pyenv/versions/yt-py310/lib/python3.10/site-packages/setuptools/command/egg_info.py", line 541, in run
        self.add_defaults()
      File "/Users/robcleme/.pyenv/versions/yt-py310/lib/python3.10/site-packages/setuptools/command/egg_info.py", line 578, in add_defaults
        sdist.add_defaults(self)
      File "/Users/robcleme/.pyenv/versions/3.10.0b2/lib/python3.10/distutils/command/sdist.py", line 228, in add_defaults
        self._add_defaults_ext()
      File "/Users/robcleme/.pyenv/versions/3.10.0b2/lib/python3.10/distutils/command/sdist.py", line 311, in _add_defaults_ext
        build_ext = self.get_finalized_command('build_ext')
      File "/Users/robcleme/.pyenv/versions/3.10.0b2/lib/python3.10/distutils/cmd.py", line 299, in get_finalized_command
        cmd_obj.ensure_finalized()
      File "/Users/robcleme/.pyenv/versions/3.10.0b2/lib/python3.10/distutils/cmd.py", line 107, in ensure_finalized
        self.finalize_options()
      File "/Users/robcleme/dev/yt-project/yt-py310/setupext.py", line 403, in finalize_options
        self.distribution.ext_modules[:] = cythonize(
      File "/Users/robcleme/.pyenv/versions/yt-py310/lib/python3.10/site-packages/Cython/Build/Dependencies.py", line 1107, in cythonize
        result.get(99999)  # seconds
      File "/Users/robcleme/.pyenv/versions/3.10.0b2/lib/python3.10/multiprocessing/pool.py", line 771, in get
        raise self._value
    Cython.Compiler.Errors.CompileError: yt/geometry/particle_smooth.pyx
    ----------------------------------------
WARNING: Discarding file:///Users/robcleme/dev/yt-project/yt-py310. Command errored out with exit status 1: /Users/robcleme/.pyenv/versions/yt-py310/bin/python /Users/robcleme/.pyenv/versions/yt-py310/lib/python3.10/site-packages/pip/_vendor/pep517/in_process/_in_process.py prepare_metadata_for_build_wheel /var/folders/xh/yfmp7l9n3x50ln2yvrndt7c801pbkm/T/tmpfctihuv8 Check the logs for full command output.
ERROR: Command errored out with exit status 1: /Users/robcleme/.pyenv/versions/yt-py310/bin/python /Users/robcleme/.pyenv/versions/yt-py310/lib/python3.10/site-packages/pip/_vendor/pep517/in_process/_in_process.py prepare_metadata_for_build_wheel /var/folders/xh/yfmp7l9n3x50ln2yvrndt7c801pbkm/T/tmpfctihuv8 Check the logs for full command output.

Code for reproduction

python -m pip install -pre cython
python -m pip install -e --no-build-isolation .

Version Information

  • Operating System: OsX
  • Python Version: 3.10.0a2 (yt is known to compile fine with it + Cython 0.29.21, so I don't think that the issue lie here.)
  • yt version: main branch (419a8eb0ddc7bcee6abcf92eed1fbbedace1d60d)

neutrinoceros avatar Jul 09 '21 17:07 neutrinoceros

Note: this is a Cython bug, which has been reported upstream (see https://github.com/cython/cython/issues/4313) and there is an open PR to fix this as well (https://github.com/cython/cython/pull/4315).

cphyc avatar Jul 28 '21 14:07 cphyc

Excellent, thank you ! Once it resolved upstream we can add Cython 3.0 to the "bleeding edge" CI job

neutrinoceros avatar Jul 28 '21 14:07 neutrinoceros

Note: switching to Cython 3 will silence deprecation warnings we get at build time from numpy, see https://github.com/cython/cython/issues/2498

neutrinoceros avatar Sep 09 '21 07:09 neutrinoceros

Think this would be a good candidate for a small sprint?

matthewturk avatar Sep 09 '21 10:09 matthewturk

not sure there's anything we can/should do actually. Right now there's a single bug we know off that's blocking us, and Corentin already opened a PR to fix it upstream, so AFAIK all we have to do is wait.

neutrinoceros avatar Sep 09 '21 12:09 neutrinoceros

@cphyc's PR was merged ! I checked that it works properly, though we probably shouldn't declare compatibility before numpy does, and they are waiting on pypy 7.3.8, see https://github.com/numpy/numpy/blob/138963bd61e4d6e82261ae8c8db98394b69fe3f5/pyproject.toml#L7

neutrinoceros avatar Dec 25 '21 16:12 neutrinoceros

This was fixed upstream

neutrinoceros avatar Jul 27 '23 21:07 neutrinoceros