pyart
pyart copied to clipboard
ARM-DOE
#No module named 'numpy.distutils._msvccompiler'
No module named 'numpy.distutils._msvccompiler' in numpy.distutils; trying from distutils customize MSVCCompiler customize MSVCCompiler using build_ext building 'pyart.io._sigmetfile' extension compiling C sources error: Unable to find vcvarsall.bat
I got this issue when i tried to build setup.py. Can anyone please help me with this issue?
@Red025 How is your numpy installed? I would recommend using conda if you aren't already. Numpy sometimes has issues building on Windows: https://github.com/numpy/numpy/issues/12016
Hmm, maybe try a fresh environment:
conda create -n pyart_env -c conda-forge python=3.7 numpy scipy netCDF4 xarray cartopy matplotlib
source activate pyart_env
then install pyart within the environment.
No module named 'numpy.distutils._msvccompiler' in numpy.distutils; trying from distutils customize MSVCCompiler customize MSVCCompiler using build_ext building 'pyart.io._sigmetfile' extension compiling C sources error: Unable to find vcvarsall.bat
Still getting same error. Is there something wrong with my Microsoft visual c++ version?
I would maybe try: python -m pip install -U pip pip install -U setuptools But my guess would be, as you mentioned, something with Microsoft visual c++, and might need to try one of the solutions found here: https://stackoverflow.com/questions/27670365/python-pip-install-error-unable-to-find-vcvarsall-bat-tried-all-solutions/30098267 @rcjackson Thoughts on this?
Are you conda installing Py-ART?
It is NOT recommended to install Py-ART from source on Windows..
conda -c conda-forge install arm_pyart
From: Red025 [email protected] Reply-To: ARM-DOE/pyart [email protected] Date: Wednesday, October 2, 2019 at 12:08 PM To: ARM-DOE/pyart [email protected] Cc: Subscribed [email protected] Subject: Re: [ARM-DOE/pyart] #No module named 'numpy.distutils._msvccompiler' (#869)
No module named 'numpy.distutils._msvccompiler' in numpy.distutils; trying from distutils customize MSVCCompiler customize MSVCCompiler using build_ext building 'pyart.io._sigmetfile' extension compiling C sources error: Unable to find vcvarsall.bat
Still getting same error. Is there something wrong with my Microsoft visual c++ version?
— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub, or mute the thread.
If you use Python 3.8 that isn't officially supported by NumPy yet. I suggest you stick with python 3.7 for now until you see a 3.8 designation in the "Programming Language" section on numpy page.
We do no testing for 3.8.. if you go there you are on your own ;)
-sent from a mobile device-
On Nov 3, 2019, at 9:26 AM, goldengrisha [email protected] wrote:
If you use Python 3.8 that isn't officially supported by NumPy yet. I suggest you stick with python 3.7 for now until you see a 3.8 designation in the "Programming Language" section of the PyPI numpy page.
— You are receiving this because you commented. Reply to this email directly, view it on GitHub, or unsubscribe.
@zhaobudaola, do you have the visual studios installed on your computer for windows, I know for windows its needed.
This post is slightly outdated, but give this a read, it might have a solution to this: https://stackoverflow.com/questions/44951456/pip-error-microsoft-visual-c-14-0-is-required
I have installed visual C++build tools and visual studio 2015, but still got this issue.
Hmm, is your numpy installed with conda or pip, i'm honestly not entirely sure how to go about this, using visual studios tends to be the error, but i'm running out of ideas
This issue is maybe because your visual studio is not coming in to rescue. I was facing the same error when installing scikit-learn in development mode. I resolved it as follows:
- First install the build tools for visual studio 2019
- Run following two commands in your anaconda prompt.
SET DISTUTILS_USE_SDK=1"C:\Program Files (x86)\Microsoft Visual Studio\2019\BuildTools\VC\Auxiliary\Build\vcvarsall.bat" x64This resolved my issue for 64 bit python version. Here is the reference link https://scikit-learn.org/dev/developers/advanced_installation.html#install-bleeding-edge
I have followed the suggestions from this link, and I am still getting problems installing the module.
I am using python 3.7.3 in a conda environment, with all dependencies installed.
(sklearn-env) E:\python\scikit-learn>SET DISTUTILS_USE_SDK=1
(sklearn-env) E:\python\scikit-learn>"C:\Program Files (x86)\Microsoft Visual St
udio\2019\Professional\VC\Auxiliary\Build\vcvarsall.bat" x64
**********************************************************************
** Visual Studio 2019 Developer Command Prompt v16.11.10
** Copyright (c) 2021 Microsoft Corporation
**********************************************************************
[vcvarsall.bat] Environment initialized for: 'x64'
(sklearn-env) E:\python\scikit-learn>python -m pip install --verbose --no-build-isolation --editable .
Using pip 21.2.4 from C:\Users\pcme\miniconda3\envs\sklearn-env\lib\site-packages\pip (python 3.7)
Obtaining file:///E:/python/scikit-learn
Running command 'C:\Users\pcme\miniconda3\envs\sklearn-env\python.exe' 'C:\Users\pcme\miniconda3\envs\sklearn-env\lib\site-packages\pip\_vendor\pep517\in_process\_in_process.py' prepare_metadata_for_build_wheel 'C:\Users\pcme\AppData\Local\Temp\tmparbub5gp'
No module named 'numpy.distutils._msvccompiler' in numpy.distutils; trying from distutils
C:\Program Files (x86)\Microsoft Visual Studio\2019\Professional\VC\Tools\MSVC\14.29.30133\bin\HostX64\x64\cl.exe /c /nologo /Ox /W3 /GL /DNDEBUG /MT -IC:\Program Files (x86)\Microsoft Visual Studio\2019\Professional\VC\Tools\MSVC\14.29.30133\include -IC:\Program Files (x86)\Windows Kits\NETFXSDK\4.8\include\um -IC:\Program Files (x86)\Windows Kits\10\include\10.0.19041.0\ucrt -IC:\Program Files (x86)\Windows Kits\10\include\10.0.19041.0\shared -IC:\Program Files (x86)\Windows Kits\10\include\10.0.19041.0\um -IC:\Program Files (x86)\Windows Kits\10\include\10.0.19041.0\winrt -IC:\Program Files (x86)\Windows Kits\10\include\10.0.19041.0\cppwinrt /Tctest_program.c /Foobjects\test_program.obj
C:\Program Files (x86)\Microsoft Visual Studio\2019\Professional\VC\Tools\MSVC\14.29.30133\bin\HostX64\x64\link.exe /nologo /INCREMENTAL:NO /LTCG /nodefaultlib:libucrt.lib ucrt.lib /MANIFEST:EMBED,ID=1 /LIBPATH:C:\Program Files (x86)\Microsoft Visual Studio\2019\Professional\VC\Tools\MSVC\14.29.30133\lib\x64 /LIBPATH:C:\Program Files (x86)\Windows Kits\NETFXSDK\4.8\lib\um\x64 /LIBPATH:C:\Program Files (x86)\Windows Kits\10\lib\10.0.19041.0\ucrt\x64 /LIBPATH:C:\Program Files (x86)\Windows Kits\10\lib\10.0.19041.0\um\x64 objects\test_program.obj /OUT:test_program.exe
Partial import of sklearn during the build process.
Traceback (most recent call last):
File "C:\Users\pcme\miniconda3\envs\sklearn-env\lib\site-packages\pip\_vendor\pep517\in_process\_in_process.py", line 349, in <module>
main()
File "C:\Users\pcme\miniconda3\envs\sklearn-env\lib\site-packages\pip\_vendor\pep517\in_process\_in_process.py", line 331, in main
json_out['return_val'] = hook(**hook_input['kwargs'])
File "C:\Users\pcme\miniconda3\envs\sklearn-env\lib\site-packages\pip\_vendor\pep517\in_process\_in_process.py", line 151, in prepare_metadata_for_build_wheel
return hook(metadata_directory, config_settings)
File "C:\Users\pcme\miniconda3\envs\sklearn-env\lib\site-packages\setuptools\build_meta.py", line 166, in prepare_metadata_for_build_wheel
self.run_setup()
File "C:\Users\pcme\miniconda3\envs\sklearn-env\lib\site-packages\setuptools\build_meta.py", line 259, in run_setup
self).run_setup(setup_script=setup_script)
File "C:\Users\pcme\miniconda3\envs\sklearn-env\lib\site-packages\setuptools\build_meta.py", line 150, in run_setup
exec(compile(code, __file__, 'exec'), locals())
File "setup.py", line 306, in <module>
setup_package()
File "setup.py", line 302, in setup_package
setup(**metadata)
File "C:\Users\pcme\miniconda3\envs\sklearn-env\lib\site-packages\numpy\distutils\core.py", line 135, in setup
config = configuration()
File "setup.py", line 187, in configuration
config.add_subpackage("sklearn")
File "C:\Users\pcme\miniconda3\envs\sklearn-env\lib\site-packages\numpy\distutils\misc_util.py", line 1016, in add_subpackage
caller_level = 2)
File "C:\Users\pcme\miniconda3\envs\sklearn-env\lib\site-packages\numpy\distutils\misc_util.py", line 985, in get_subpackage
caller_level = caller_level + 1)
File "C:\Users\pcme\miniconda3\envs\sklearn-env\lib\site-packages\numpy\distutils\misc_util.py", line 922, in _get_configuration_from_setup_py
config = setup_module.configuration(*args)
File "sklearn\setup.py", line 85, in configuration
cythonize_extensions(top_path, config)
File "E:\python\scikit-learn\sklearn\_build_utils\__init__.py", line 47, in cythonize_extensions
basic_check_build()
File "E:\python\scikit-learn\sklearn\_build_utils\pre_build_helpers.py", line 119, in basic_check_build
compile_test_program(code)
File "E:\python\scikit-learn\sklearn\_build_utils\pre_build_helpers.py", line 100, in compile_test_program
os.chdir(start_dir)
File "C:\Users\pcme\miniconda3\envs\sklearn-env\lib\tempfile.py", line 805, in __exit__
self.cleanup()
File "C:\Users\pcme\miniconda3\envs\sklearn-env\lib\tempfile.py", line 809, in cleanup
_shutil.rmtree(self.name)
File "C:\Users\pcme\miniconda3\envs\sklearn-env\lib\shutil.py", line 513, in rmtree
return _rmtree_unsafe(path, onerror)
File "C:\Users\pcme\miniconda3\envs\sklearn-env\lib\shutil.py", line 401, in _rmtree_unsafe
onerror(os.rmdir, path, sys.exc_info())
File "C:\Users\pcme\miniconda3\envs\sklearn-env\lib\shutil.py", line 399, in _rmtree_unsafe
os.rmdir(path)
OSError: [WinError 145] The directory is not empty: 'C:\\Users\\pcme\\AppData\\Local\\Temp\\tmpvx99h2bb'
Preparing wheel metadata ... error
WARNING: Discarding file:///E:/python/scikit-learn. Command errored out with exit status 1: 'C:\Users\pcme\miniconda3\envs\sklearn-env\python.exe' 'C:\Users\pcme\miniconda3\envs\sklearn-env\lib\site-packages\pip\_vendor\pep517\in_process\_in_process.py' prepare_metadata_for_build_wheel 'C:\Users\pcme\AppData\Local\Temp\tmparbub5gp' Check the logs for full command output.
ERROR: Command errored out with exit status 1: 'C:\Users\pcme\miniconda3\envs\sklearn-env\python.exe' 'C:\Users\pcme\miniconda3\envs\sklearn-env\lib\site-packages\pip\_vendor\pep517\in_process\_in_process.py' prepare_metadata_for_build_wheel 'C:\Users\pcme\AppData\Local\Temp\tmparbub5gp' Check the logs for full command output.
Are there any other suggestions I could try?
@Jillinger I'm honestly not sure. If trying all the options above didn't work i'm honestly out of ideas with this one. If you are trying to install for scikit learn, I would ping their issue board on their installation method for scikit learn or their community board. Make sure you have an up to date visual studios and python with the path. Are any other packages working in that environment?
Yes, all packages installed nicely, to my surprise, since I had trouble installing a few, like osmium. I was trying to install pyrosm, which requires some dependencies I am having trouble with, but yes, my Visual Studio is up to date, as well as the Build Tools, and I set my portable Python in the Path. Everything works to my satisfaction... except the above mentioned error. I tried every suggestion I could find.
If I could get the pyrosm wheel, that would be nice. If it was not dangerous, I would request someone "pip wheel pyrosm" and send me the file. Edit: But then again, that might still require the dependencies. I just wanted to try it out. If I can't, I will just work with what I have at present. Thanks
@zssherman I think I found a way around the problem - creating wheels.
python -m pip wheel --wheel-dir=E:\python\myWhls scikit-learn
Collecting scikit-learn
Downloading scikit_learn-1.0.2-cp37-cp37m-win_amd64.whl (7.1 MB)
---------------------------------------- 7.1/7.1 MB 6.3 MB/s eta 0:00:00
Collecting numpy>=1.14.6
Using cached numpy-1.21.5-cp37-cp37m-win_amd64.whl (14.0 MB)
Collecting scipy>=1.1.0
Using cached scipy-1.7.3-cp37-cp37m-win_amd64.whl (34.1 MB)
Collecting joblib>=0.11
Downloading joblib-1.1.0-py2.py3-none-any.whl (306 kB)
-------------------------------------- 307.0/307.0 KB 3.8 MB/s eta 0:00:00
Collecting threadpoolctl>=2.0.0
Using cached threadpoolctl-3.1.0-py3-none-any.whl (14 kB)
Saved e:\python\mywhls\scikit_learn-1.0.2-cp37-cp37m-win_amd64.whl
Saved e:\python\mywhls\joblib-1.1.0-py2.py3-none-any.whl
Saved e:\python\mywhls\numpy-1.21.5-cp37-cp37m-win_amd64.whl
Saved e:\python\mywhls\scipy-1.7.3-cp37-cp37m-win_amd64.whl
Saved e:\python\mywhls\threadpoolctl-3.1.0-py3-none-any.whl
python -m pip install E:\python\myWhls\scikit_learn-1.0.2-cp37-cp37m-win_amd64.whl
Processing e:\python\mywhls\scikit_learn-1.0.2-cp37-cp37m-win_amd64.whl
Requirement already satisfied: threadpoolctl>=2.0.0 in e:\python\python37\portab
lepython3.7.3x64\app\python\lib\site-packages (from scikit-learn==1.0.2) (3.1.0)
Requirement already satisfied: numpy>=1.14.6 in e:\python\python37\portablepytho
n3.7.3x64\app\python\lib\site-packages (from scikit-learn==1.0.2) (1.21.5+mkl)
Requirement already satisfied: joblib>=0.11 in e:\python\python37\portablepython
3.7.3x64\app\python\lib\site-packages (from scikit-learn==1.0.2) (1.1.0)
Requirement already satisfied: scipy>=1.1.0 in e:\python\python37\portablepython
3.7.3x64\app\python\lib\site-packages (from scikit-learn==1.0.2) (1.7.3)
Installing collected packages: scikit-learn
Successfully installed scikit-learn-1.0.2
One question though. How do I clear the cache for pip? I am trying to install a wheel, but pip is using the cache, and building the wheel results in an error. No worries. I found the answer. --no-cache-dir
Leaving this open as a reference for other users who might be running into similar issues.