securedrop-docs
securedrop-docs copied to clipboard
freedomofpress
development-requirements installation fails due to missing python-dev package
Description
Following the installation instructions for the development environment the command to install the development-requirements.txtfails.
Steps to Reproduce
Fresh install of the development environment as instructed up to this step and run the following, as instructed
pip install -r securedrop/requirements/python3/develop-requirements.txt
Expected Behavior
Command to run without errors
Actual Behavior
Building wheel for port-for (setup.py) ... done
Created wheel for port-for: filename=port_for-0.3.1-cp35-none-any.whl size=19607 sha256=075597faf2545f46e10be8d8e3ecb8a800a7a0297bd7a00847b694b65be301ba
Stored in directory: /home/user/.cache/pip/wheels/76/d3/1b/ea48e3544c50666eed11eac26df8c741d197106ded6fd646e3
Building wheel for psutil (setup.py) ... error
ERROR: Command errored out with exit status 1:
command: /home/user/.virtualenvs/securedrop/bin/python3 -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-hoy97m7e/psutil/setup.py'"'"'; __file__='"'"'/tmp/pip-install-hoy97m7e/psutil/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' bdist_wheel -d /tmp/pip-wheel-uxmjdzbw --python-tag cp35
cwd: /tmp/pip-install-hoy97m7e/psutil/
Complete output (44 lines):
running bdist_wheel
running build
running build_py
creating build
creating build/lib.linux-x86_64-3.5
creating build/lib.linux-x86_64-3.5/psutil
copying psutil/_pslinux.py -> build/lib.linux-x86_64-3.5/psutil
copying psutil/_psposix.py -> build/lib.linux-x86_64-3.5/psutil
copying psutil/_psaix.py -> build/lib.linux-x86_64-3.5/psutil
copying psutil/_pssunos.py -> build/lib.linux-x86_64-3.5/psutil
copying psutil/_pswindows.py -> build/lib.linux-x86_64-3.5/psutil
copying psutil/_common.py -> build/lib.linux-x86_64-3.5/psutil
copying psutil/_compat.py -> build/lib.linux-x86_64-3.5/psutil
copying psutil/_exceptions.py -> build/lib.linux-x86_64-3.5/psutil
copying psutil/__init__.py -> build/lib.linux-x86_64-3.5/psutil
copying psutil/_psosx.py -> build/lib.linux-x86_64-3.5/psutil
copying psutil/_psbsd.py -> build/lib.linux-x86_64-3.5/psutil
creating build/lib.linux-x86_64-3.5/psutil/tests
copying psutil/tests/test_posix.py -> build/lib.linux-x86_64-3.5/psutil/tests
copying psutil/tests/test_osx.py -> build/lib.linux-x86_64-3.5/psutil/tests
copying psutil/tests/test_system.py -> build/lib.linux-x86_64-3.5/psutil/tests
copying psutil/tests/test_misc.py -> build/lib.linux-x86_64-3.5/psutil/tests
copying psutil/tests/test_bsd.py -> build/lib.linux-x86_64-3.5/psutil/tests
copying psutil/tests/__main__.py -> build/lib.linux-x86_64-3.5/psutil/tests
copying psutil/tests/test_unicode.py -> build/lib.linux-x86_64-3.5/psutil/tests
copying psutil/tests/__init__.py -> build/lib.linux-x86_64-3.5/psutil/tests
copying psutil/tests/test_process.py -> build/lib.linux-x86_64-3.5/psutil/tests
copying psutil/tests/test_contracts.py -> build/lib.linux-x86_64-3.5/psutil/tests
copying psutil/tests/test_linux.py -> build/lib.linux-x86_64-3.5/psutil/tests
copying psutil/tests/test_sunos.py -> build/lib.linux-x86_64-3.5/psutil/tests
copying psutil/tests/test_windows.py -> build/lib.linux-x86_64-3.5/psutil/tests
copying psutil/tests/test_aix.py -> build/lib.linux-x86_64-3.5/psutil/tests
copying psutil/tests/test_memory_leaks.py -> build/lib.linux-x86_64-3.5/psutil/tests
copying psutil/tests/test_connections.py -> build/lib.linux-x86_64-3.5/psutil/tests
running build_ext
building 'psutil._psutil_linux' extension
creating build/temp.linux-x86_64-3.5
creating build/temp.linux-x86_64-3.5/psutil
x86_64-linux-gnu-gcc -pthread -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -g -fdebug-prefix-map=/build/python3.5-3.5.3=. -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 -fPIC -DPSUTIL_POSIX=1 -DPSUTIL_VERSION=546 -DPSUTIL_LINUX=1 -I/usr/include/python3.5m -I/home/user/.virtualenvs/securedrop/include/python3.5m -c psutil/_psutil_common.c -o build/temp.linux-x86_64-3.5/psutil/_psutil_common.o
psutil/_psutil_common.c:9:20: fatal error: Python.h: No such file or directory
#include <Python.h>
^
compilation terminated.
error: command 'x86_64-linux-gnu-gcc' failed with exit status 1
----------------------------------------
ERROR: Failed building wheel for psutil
Running setup.py clean for psutil
Building wheel for pyenchant (setup.py) ... done
[...]
Failed to build psutil
Installing collected packages: alabaster, markupsafe, jinja2, pyyaml, pycparser, cffi, six, bcrypt, pyasn1, asn1crypto, cryptography, pynacl, paramiko, ansible, ruamel.yaml, ansible-lint, anyconfig, apipkg, argh, python-dateutil, arrow, lazy-object-proxy, wrapt, astroid, pytz, babel, smmap2, gitdb2, gitpython, pbr, stevedore, bandit, chardet, binaryornot, jmespath, docutils, botocore, s3transfer, boto3, boto, cerberus, certifi, click, click-completion, colorama, idna, urllib3, requests, jinja2-time, future, poyo, whichcraft, cookiecutter, dnspython, websocket-client, docker-pycreds, docker-py, docopt, pyparsing, packaging, dparse, entrypoints, execnet, monotonic, fasteners, pyflakes, mccabe, pycodestyle, flake8, git-url-parse, template-remover, html-linter, imagesize, isort, tornado, livereload, sh, psutil, pathspec, yamllint, py, pytest, testinfra, tree-format, ptyprocess, pexpect, tabulate, python-gilt, molecule, mypy-extensions, typed-ast, mypy, netaddr, pathtools, pip-tools, port-for, wcwidth, prompt-toolkit, pyenchant, pygments, pylint, pytest-xdist, python-vagrant, safety, snowballstemmer, watchdog, sphinx-autobuild, sphinx-rtd-theme, sphinxcontrib-websupport, sphinx
Running setup.py install for psutil ... error
ERROR: Command errored out with exit status 1:
command: /home/user/.virtualenvs/securedrop/bin/python3 -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-hoy97m7e/psutil/setup.py'"'"'; __file__='"'"'/tmp/pip-install-hoy97m7e/psutil/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' install --record /tmp/pip-record-b9ii7lwl/install-record.txt --single-version-externally-managed --compile --install-headers /home/user/.virtualenvs/securedrop/include/site/python3.5/psutil
cwd: /tmp/pip-install-hoy97m7e/psutil/
Complete output (44 lines):
running install
running build
running build_py
creating build
creating build/lib.linux-x86_64-3.5
creating build/lib.linux-x86_64-3.5/psutil
copying psutil/_pslinux.py -> build/lib.linux-x86_64-3.5/psutil
copying psutil/_psposix.py -> build/lib.linux-x86_64-3.5/psutil
copying psutil/_psaix.py -> build/lib.linux-x86_64-3.5/psutil
copying psutil/_pssunos.py -> build/lib.linux-x86_64-3.5/psutil
copying psutil/_pswindows.py -> build/lib.linux-x86_64-3.5/psutil
copying psutil/_common.py -> build/lib.linux-x86_64-3.5/psutil
copying psutil/_compat.py -> build/lib.linux-x86_64-3.5/psutil
copying psutil/_exceptions.py -> build/lib.linux-x86_64-3.5/psutil
copying psutil/__init__.py -> build/lib.linux-x86_64-3.5/psutil
copying psutil/_psosx.py -> build/lib.linux-x86_64-3.5/psutil
copying psutil/_psbsd.py -> build/lib.linux-x86_64-3.5/psutil
creating build/lib.linux-x86_64-3.5/psutil/tests
copying psutil/tests/test_posix.py -> build/lib.linux-x86_64-3.5/psutil/tests
copying psutil/tests/test_osx.py -> build/lib.linux-x86_64-3.5/psutil/tests
copying psutil/tests/test_system.py -> build/lib.linux-x86_64-3.5/psutil/tests
copying psutil/tests/test_misc.py -> build/lib.linux-x86_64-3.5/psutil/tests
copying psutil/tests/test_bsd.py -> build/lib.linux-x86_64-3.5/psutil/tests
copying psutil/tests/__main__.py -> build/lib.linux-x86_64-3.5/psutil/tests
copying psutil/tests/test_unicode.py -> build/lib.linux-x86_64-3.5/psutil/tests
copying psutil/tests/__init__.py -> build/lib.linux-x86_64-3.5/psutil/tests
copying psutil/tests/test_process.py -> build/lib.linux-x86_64-3.5/psutil/tests
copying psutil/tests/test_contracts.py -> build/lib.linux-x86_64-3.5/psutil/tests
copying psutil/tests/test_linux.py -> build/lib.linux-x86_64-3.5/psutil/tests
copying psutil/tests/test_sunos.py -> build/lib.linux-x86_64-3.5/psutil/tests
copying psutil/tests/test_windows.py -> build/lib.linux-x86_64-3.5/psutil/tests
copying psutil/tests/test_aix.py -> build/lib.linux-x86_64-3.5/psutil/tests
copying psutil/tests/test_memory_leaks.py -> build/lib.linux-x86_64-3.5/psutil/tests
copying psutil/tests/test_connections.py -> build/lib.linux-x86_64-3.5/psutil/tests
running build_ext
building 'psutil._psutil_linux' extension
creating build/temp.linux-x86_64-3.5
creating build/temp.linux-x86_64-3.5/psutil
x86_64-linux-gnu-gcc -pthread -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -g -fdebug-prefix-map=/build/python3.5-3.5.3=. -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 -fPIC -DPSUTIL_POSIX=1 -DPSUTIL_VERSION=546 -DPSUTIL_LINUX=1 -I/usr/include/python3.5m -I/home/user/.virtualenvs/securedrop/include/python3.5m -c psutil/_psutil_common.c -o build/temp.linux-x86_64-3.5/psutil/_psutil_common.o
psutil/_psutil_common.c:9:20: fatal error: Python.h: No such file or directory
#include <Python.h>
^
compilation terminated.
error: command 'x86_64-linux-gnu-gcc' failed with exit status 1
----------------------------------------
ERROR: Command errored out with exit status 1: /home/user/.virtualenvs/securedrop/bin/python3 -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-hoy97m7e/psutil/setup.py'"'"'; __file__='"'"'/tmp/pip-install-hoy97m7e/psutil/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' install --record /tmp/pip-record-b9ii7lwl/install-record.txt --single-version-externally-managed --compile --install-headers /home/user/.virtualenvs/securedrop/include/site/python3.5/psutil Check the logs for full command output.
(successful installations omitted with [...] )
Comments
The solution is to install the development version of python (as advised in stackoverflow).
To avoid this mistake the docs could mention as a warning that the user should install the development version of python if the command fails.
Needs a repro on current develop, if this can still arise, we should indeed document the dependency.
All of the Python packages we depend on that have compiled code ship binary wheels for most platforms. I'm going to close this for now but if there's a specific dependency that is missing wheels for a common development platform we can figure something out.
