crossenv icon indicating copy to clipboard operation
crossenv copied to clipboard
verified publisher icon benfogle

cross compile numpy in mips platform error

Open leidemon opened this issue 5 years ago • 4 comments

hi, crossenv team: I want to cross compile numpy-1.19.4 in mips platform, and I have the issue: Running from numpy source directory.

Note: if you need reliable uninstall behavior, then install with pip instead of using setup.py install:

  • pip install . (from a git repo or downloaded source release)
  • pip install numpy (last NumPy release on PyPi)

Cythonizing sources numpy/random/_bounded_integers.pxd.in has not changed numpy/random/bit_generator.pyx has not changed numpy/random/_common.pyx has not changed numpy/random/_mt19937.pyx has not changed numpy/random/mtrand.pyx has not changed numpy/random/_pcg64.pyx has not changed numpy/random/_philox.pyx has not changed numpy/random/_sfc64.pyx has not changed numpy/random/_bounded_integers.pyx.in has not changed numpy/random/_bounded_integers.pyx has not changed numpy/random/_generator.pyx has not changed blas_opt_info: blas_mkl_info: customize UnixCCompiler C compiler: ccache mipsel-linux-uclibc-gcc -Os -pipe -mips32 -mtune=mips32 -funit-at-a-time -Wno-unused-result -Wsign-compare -Os -pipe -mips32 -mtune=mips32 -funit-at-a-time -I/home/lwd/py3/src/router/newopenssl/include -I/home/lwd/py3/src/router/zlib -Os -pipe -mips32 -mtune=mips32 -funit-at-a-time -I/home/lwd/py3/src/router/newopenssl/include -I/home/lwd/py3/src/router/zlib -I/home/lwd/py3/src/router/mipsel-uclibc/install/libffi/usr/lib/libffi-3.2.1/include -Os -pipe -mips32 -mtune=mips32 -funit-at-a-time -fPIC

creating /tmp/tmp1r9ipjim/tmp creating /tmp/tmp1r9ipjim/tmp/tmp1r9ipjim compile options: '-MMD -MF /tmp/tmp1r9ipjim/file.c.d -c' ccache: /tmp/tmp1r9ipjim/file.c libraries mkl_rt not found in ['/home/lwd/python_lib/test/cross_venv/cross/lib', '/usr/local/lib', '/usr/lib', '/usr/lib/i386-linux-gnu'] NOT AVAILABLE

blis_info: libraries blis not found in ['/home/lwd/python_lib/test/cross_venv/cross/lib', '/usr/local/lib', '/usr/lib', '/usr/lib/i386-linux-gnu'] NOT AVAILABLE

openblas_info: libraries openblas not found in ['/home/lwd/python_lib/test/cross_venv/cross/lib', '/usr/local/lib', '/usr/lib', '/usr/lib/i386-linux-gnu'] NOT AVAILABLE

atlas_3_10_blas_threads_info: Setting PTATLAS=ATLAS libraries tatlas not found in ['/home/lwd/python_lib/test/cross_venv/cross/lib', '/usr/local/lib', '/usr/lib/sse2', '/usr/lib', '/usr/lib/i386-linux-gnu/sse2', '/usr/lib/i386-linux-gnu'] NOT AVAILABLE

atlas_3_10_blas_info: libraries satlas not found in ['/home/lwd/python_lib/test/cross_venv/cross/lib', '/usr/local/lib', '/usr/lib/sse2', '/usr/lib', '/usr/lib/i386-linux-gnu/sse2', '/usr/lib/i386-linux-gnu'] NOT AVAILABLE

atlas_blas_threads_info: Setting PTATLAS=ATLAS libraries ptf77blas,ptcblas,atlas not found in ['/home/lwd/python_lib/test/cross_venv/cross/lib', '/usr/local/lib', '/usr/lib/sse2', '/usr/lib', '/usr/lib/i386-linux-gnu/sse2', '/usr/lib/i386-linux-gnu'] NOT AVAILABLE

atlas_blas_info: libraries f77blas,cblas,atlas not found in ['/home/lwd/python_lib/test/cross_venv/cross/lib', '/usr/local/lib', '/usr/lib/sse2', '/usr/lib', '/usr/lib/i386-linux-gnu/sse2', '/usr/lib/i386-linux-gnu'] NOT AVAILABLE

accelerate_info: NOT AVAILABLE

/home/lwd/python_lib/numpy-1.19.4/numpy/distutils/system_info.py:1914: UserWarning: Optimized (vendor) Blas libraries are not found. Falls back to netlib Blas library which has worse performance. A better performance should be easily gained by switching Blas library. if self._calc_info(blas): blas_info: libraries blas not found in ['/home/lwd/python_lib/test/cross_venv/cross/lib', '/usr/local/lib', '/usr/lib', '/usr/lib/i386-linux-gnu'] NOT AVAILABLE

/home/lwd/python_lib/numpy-1.19.4/numpy/distutils/system_info.py:1914: UserWarning: Blas (http://www.netlib.org/blas/) libraries not found. Directories to search for the libraries can be specified in the numpy/distutils/site.cfg file (section [blas]) or by setting the BLAS environment variable. if self._calc_info(blas): blas_src_info: NOT AVAILABLE

/home/lwd/python_lib/numpy-1.19.4/numpy/distutils/system_info.py:1914: UserWarning: Blas (http://www.netlib.org/blas/) sources not found. Directories to search for the sources can be specified in the numpy/distutils/site.cfg file (section [blas_src]) or by setting the BLAS_SRC environment variable. if self._calc_info(blas): NOT AVAILABLE

non-existing path in 'numpy/distutils': 'site.cfg' lapack_opt_info: lapack_mkl_info: libraries mkl_rt not found in ['/home/lwd/python_lib/test/cross_venv/cross/lib', '/usr/local/lib', '/usr/lib', '/usr/lib/i386-linux-gnu'] NOT AVAILABLE

openblas_lapack_info: libraries openblas not found in ['/home/lwd/python_lib/test/cross_venv/cross/lib', '/usr/local/lib', '/usr/lib', '/usr/lib/i386-linux-gnu'] NOT AVAILABLE

openblas_clapack_info: libraries openblas,lapack not found in ['/home/lwd/python_lib/test/cross_venv/cross/lib', '/usr/local/lib', '/usr/lib', '/usr/lib/i386-linux-gnu'] NOT AVAILABLE

flame_info: libraries flame not found in ['/home/lwd/python_lib/test/cross_venv/cross/lib', '/usr/local/lib', '/usr/lib', '/usr/lib/i386-linux-gnu'] NOT AVAILABLE

atlas_3_10_threads_info: Setting PTATLAS=ATLAS libraries lapack_atlas not found in /home/lwd/python_lib/test/cross_venv/cross/lib libraries tatlas,tatlas not found in /home/lwd/python_lib/test/cross_venv/cross/lib libraries lapack_atlas not found in /usr/local/lib libraries tatlas,tatlas not found in /usr/local/lib libraries lapack_atlas not found in /usr/lib/sse2 libraries tatlas,tatlas not found in /usr/lib/sse2 libraries lapack_atlas not found in /usr/lib libraries tatlas,tatlas not found in /usr/lib libraries lapack_atlas not found in /usr/lib/i386-linux-gnu/sse2 libraries tatlas,tatlas not found in /usr/lib/i386-linux-gnu/sse2 libraries lapack_atlas not found in /usr/lib/i386-linux-gnu libraries tatlas,tatlas not found in /usr/lib/i386-linux-gnu <class 'numpy.distutils.system_info.atlas_3_10_threads_info'> NOT AVAILABLE

atlas_3_10_info: libraries lapack_atlas not found in /home/lwd/python_lib/test/cross_venv/cross/lib libraries satlas,satlas not found in /home/lwd/python_lib/test/cross_venv/cross/lib libraries lapack_atlas not found in /usr/local/lib libraries satlas,satlas not found in /usr/local/lib libraries lapack_atlas not found in /usr/lib/sse2 libraries satlas,satlas not found in /usr/lib/sse2 libraries lapack_atlas not found in /usr/lib libraries satlas,satlas not found in /usr/lib libraries lapack_atlas not found in /usr/lib/i386-linux-gnu/sse2 libraries satlas,satlas not found in /usr/lib/i386-linux-gnu/sse2 libraries lapack_atlas not found in /usr/lib/i386-linux-gnu libraries satlas,satlas not found in /usr/lib/i386-linux-gnu <class 'numpy.distutils.system_info.atlas_3_10_info'> NOT AVAILABLE

atlas_threads_info: Setting PTATLAS=ATLAS libraries lapack_atlas not found in /home/lwd/python_lib/test/cross_venv/cross/lib libraries ptf77blas,ptcblas,atlas not found in /home/lwd/python_lib/test/cross_venv/cross/lib libraries lapack_atlas not found in /usr/local/lib libraries ptf77blas,ptcblas,atlas not found in /usr/local/lib libraries lapack_atlas not found in /usr/lib/sse2 libraries ptf77blas,ptcblas,atlas not found in /usr/lib/sse2 libraries lapack_atlas not found in /usr/lib libraries ptf77blas,ptcblas,atlas not found in /usr/lib libraries lapack_atlas not found in /usr/lib/i386-linux-gnu/sse2 libraries ptf77blas,ptcblas,atlas not found in /usr/lib/i386-linux-gnu/sse2 libraries lapack_atlas not found in /usr/lib/i386-linux-gnu libraries ptf77blas,ptcblas,atlas not found in /usr/lib/i386-linux-gnu <class 'numpy.distutils.system_info.atlas_threads_info'> NOT AVAILABLE

atlas_info: libraries lapack_atlas not found in /home/lwd/python_lib/test/cross_venv/cross/lib libraries f77blas,cblas,atlas not found in /home/lwd/python_lib/test/cross_venv/cross/lib libraries lapack_atlas not found in /usr/local/lib libraries f77blas,cblas,atlas not found in /usr/local/lib libraries lapack_atlas not found in /usr/lib/sse2 libraries f77blas,cblas,atlas not found in /usr/lib/sse2 libraries lapack_atlas not found in /usr/lib libraries f77blas,cblas,atlas not found in /usr/lib libraries lapack_atlas not found in /usr/lib/i386-linux-gnu/sse2 libraries f77blas,cblas,atlas not found in /usr/lib/i386-linux-gnu/sse2 libraries lapack_atlas not found in /usr/lib/i386-linux-gnu libraries f77blas,cblas,atlas not found in /usr/lib/i386-linux-gnu <class 'numpy.distutils.system_info.atlas_info'> NOT AVAILABLE

lapack_info: libraries lapack not found in ['/home/lwd/python_lib/test/cross_venv/cross/lib', '/usr/local/lib', '/usr/lib', '/usr/lib/i386-linux-gnu'] NOT AVAILABLE

/home/lwd/python_lib/numpy-1.19.4/numpy/distutils/system_info.py:1748: UserWarning: Lapack (http://www.netlib.org/lapack/) libraries not found. Directories to search for the libraries can be specified in the numpy/distutils/site.cfg file (section [lapack]) or by setting the LAPACK environment variable. return getattr(self, 'calc_info{}'.format(name))() lapack_src_info: NOT AVAILABLE

/home/lwd/python_lib/numpy-1.19.4/numpy/distutils/system_info.py:1748: UserWarning: Lapack (http://www.netlib.org/lapack/) sources not found. Directories to search for the sources can be specified in the numpy/distutils/site.cfg file (section [lapack_src]) or by setting the LAPACK_SRC environment variable. return getattr(self, 'calc_info{}'.format(name))() NOT AVAILABLE

numpy_linalg_lapack_lite: FOUND: language = c

/usr/local/lib/python3.8/distutils/dist.py:274: UserWarning: Unknown distribution option: 'define_macros' warnings.warn(msg) running install Checking .pth file support in /home/lwd/python_lib/tmp/lib/python3.8/site-packages /home/lwd/python_lib/test/cross_venv/cross/bin/python3.8 -E -c pass TEST FAILED: /home/lwd/python_lib/tmp/lib/python3.8/site-packages does NOT support .pth files bad install directory or PYTHONPATH

You are attempting to install a package to a directory that is not on PYTHONPATH and which Python does not read ".pth" files from. The installation directory you specified (via --install-dir, --prefix, or the distutils default setting) was:

/home/lwd/python_lib/tmp/lib/python3.8/site-packages

and your PYTHONPATH environment variable currently contains:

'/home/lwd/python_lib/test/cross_venv/lib:/usr/local/lib/python3.8'

Here are some of your options for correcting the problem:

  • You can choose a different installation directory, i.e., one that is on PYTHONPATH or supports .pth files

  • You can add the installation directory to the PYTHONPATH environment variable. (It must then also be on PYTHONPATH whenever you run Python and want to use the package(s) you are installing.)

  • You can set up the installation directory to support ".pth" files by using one of the approaches described here:

    https://setuptools.readthedocs.io/en/latest/easy_install.html#custom-installation-locations

Please make the appropriate changes for your system and try again. running bdist_egg running egg_info running build_src build_src building py_modules sources building library "npymath" sources Could not locate executable gfortran Could not locate executable f95 Could not locate executable ifort Could not locate executable ifc Could not locate executable lf95 Could not locate executable pgfortran Could not locate executable nvfortran Could not locate executable f90 Could not locate executable f77 Could not locate executable fort Could not locate executable efort Could not locate executable efc Could not locate executable g77 Could not locate executable g95 Could not locate executable pathf95 Could not locate executable nagfor don't know how to compile Fortran code on platform 'posix' adding 'build/src.linux-mipsel-3.8/numpy/core/src/npymath' to include_dirs. None - nothing done with h_files = ['build/src.linux-mipsel-3.8/numpy/core/src/npymath/npy_math_internal.h'] building library "npysort" sources adding 'build/src.linux-mipsel-3.8/numpy/core/src/common' to include_dirs. None - nothing done with h_files = ['build/src.linux-mipsel-3.8/numpy/core/src/common/npy_sort.h', 'build/src.linux-mipsel-3.8/numpy/core/src/common/npy_partition.h', 'build/src.linux-mipsel-3.8/numpy/core/src/common/npy_binsearch.h'] building library "npyrandom" sources building extension "numpy.core._multiarray_tests" sources building extension "numpy.core._multiarray_umath" sources Traceback (most recent call last): File "setup.py", line 508, in setup_package() File "setup.py", line 500, in setup_package setup(**metadata) File "/home/lwd/python_lib/numpy-1.19.4/numpy/distutils/core.py", line 169, in setup return old_setup(**new_attr) File "/home/lwd/python_lib/test/cross_venv/build/lib/python3.8/site-packages/setuptools/init.py", line 144, in setup return distutils.core.setup(**attrs) File "/usr/local/lib/python3.8/distutils/core.py", line 148, in setup dist.run_commands() File "/usr/local/lib/python3.8/distutils/dist.py", line 966, in run_commands self.run_command(cmd) File "/usr/local/lib/python3.8/distutils/dist.py", line 985, in run_command cmd_obj.run() File "/home/lwd/python_lib/numpy-1.19.4/numpy/distutils/command/install.py", line 60, in run r = self.setuptools_run() File "/home/lwd/python_lib/numpy-1.19.4/numpy/distutils/command/install.py", line 54, in setuptools_run self.do_egg_install() File "/home/lwd/python_lib/test/cross_venv/build/lib/python3.8/site-packages/setuptools/command/install.py", line 109, in do_egg_install self.run_command('bdist_egg') File "/usr/local/lib/python3.8/distutils/cmd.py", line 313, in run_command self.distribution.run_command(command) File "/usr/local/lib/python3.8/distutils/dist.py", line 985, in run_command cmd_obj.run() File "/home/lwd/python_lib/test/cross_venv/build/lib/python3.8/site-packages/setuptools/command/bdist_egg.py", line 164, in run self.run_command("egg_info") File "/usr/local/lib/python3.8/distutils/cmd.py", line 313, in run_command self.distribution.run_command(command) File "/usr/local/lib/python3.8/distutils/dist.py", line 985, in run_command cmd_obj.run() File "/home/lwd/python_lib/numpy-1.19.4/numpy/distutils/command/egg_info.py", line 24, in run self.run_command("build_src") File "/usr/local/lib/python3.8/distutils/cmd.py", line 313, in run_command self.distribution.run_command(command) File "/usr/local/lib/python3.8/distutils/dist.py", line 985, in run_command cmd_obj.run() File "/home/lwd/python_lib/numpy-1.19.4/numpy/distutils/command/build_src.py", line 144, in run self.build_sources() File "/home/lwd/python_lib/numpy-1.19.4/numpy/distutils/command/build_src.py", line 161, in build_sources self.build_extension_sources(ext) File "/home/lwd/python_lib/numpy-1.19.4/numpy/distutils/command/build_src.py", line 318, in build_extension_sources sources = self.generate_sources(sources, ext) File "/home/lwd/python_lib/numpy-1.19.4/numpy/distutils/command/build_src.py", line 378, in generate_sources source = func(extension, build_dir) File "numpy/core/setup.py", line 436, in generate_config_h moredefs, ignored = cocache.check_types(config_cmd, ext, build_dir) File "numpy/core/setup.py", line 47, in check_types out = check_types(*a, **kw) File "numpy/core/setup.py", line 292, in check_types raise SystemError( SystemError: Cannot compile 'Python.h'. Perhaps you need to install python-dev|python-devel.

I try to apt install python3-dev and pypy,but it does not works. can you help me for this issue? thank you.

leidemon avatar Dec 17 '20 12:12 leidemon

It looks like you're using the system version of Python as your build-python. It's highly recommended that you build both build-python and host-python. The version of Python that comes with your Linux distribution may not be the exact same version that you are building, and it is often patched in ways that can cause strange errors.

benfogle avatar Dec 18 '20 16:12 benfogle

看起来您正在使用系统版本的Python作为构建python。强烈建议您同时构建build-python和host-python。Linux发行版随附的Python版本可能与您所构建的版本不完全相同，并且通常以可能引起奇怪错误的方式对其进行了修补。

so how to do next? I build eth crossenv and download the numpy-1.19.4, steps as followed:

1、/host_path/python3.8(ubuntu) -m crossenv /cross_path/python3.8(mips) cross_venv 2、. ./cross_venv/bin/activate 3、python3.8 setup.py install (exec the cmd in the numpy )

leidemon avatar Dec 22 '20 03:12 leidemon

Do you mean that step 3 succeeded? In that case, you should find the files you need in cross_venv/cross/lib/python3.8/site-packages. You can copy them from there to your device to see if they work.

If you mean that step 3 failed, then I would need more information to help.

benfogle avatar Dec 24 '20 21:12 benfogle

您是说第3步成功了吗？在这种情况下，您应该在cross_venv / cross / lib / python3.8 / site-packages中找到所需的文件。您可以将它们从那里复制到设备上，以查看它们是否起作用。

如果您认为第3步失败，那么我需要更多信息来帮助您。

no, it is failed. I can not found the file in the cross_venv/cross/lib/python3.8/site-packages . I download the numpy-1.19.4 by manual, and exce the cmd in the root directory of numpy. what infos should I share for you ?

leidemon avatar Dec 25 '20 08:12 leidemon