ECS-CommunityEdition
ECS-CommunityEdition copied to clipboard
Published
20 hours ago •
EMCECS
EMCECS
'step1' fails after ecs-install built from sources
trafficstars
Expected Behavior
Installation should succeed without error.
Actual Behavior
Installation fails very early on in the 'step1' process ("Initializing data container, one moment.."). I'm having to rebuild ecs-install to work around another issue (#456) where the installation fails on NVMe storage devices due to an invalid assumption about the name of the first partition.
Steps to Reproduce Behavior
- ./bootstrap.sh --build-from http://dl-cdn.alpinelinux.org/alpine/
- step1
Relevant Output and Logs
[m4madmin@luna** ECS-CommunityEdition]$ ./bootstrap.sh --build-from http://dl-cdn.alpinelinux.org/alpine/
>
> ECS Community Edition Install Node Bootstrap 2.9.2r
> ECS Software Image emccorp/ecs-software-3.4.0:3.4.0.0
>
>
> Escalating privileges
> You may be presented with the system sudo banner and asked
> for your password depending on the Linux flavor and default
> sudo configuration for your system.
>
>
> Environment is centos linux release 7.7.1908 (core)
> [supported: true]
>
> We collect some hardware and OS info into a log file on the
> install node in case something fails and you want help trouble-
> shooting. HOWEVER, absolutely nothing is transmitted over the
> Internet or shared with EMC, GitHub, or anyone else unless or
> until you decide to attach the log file or copy & paste its
> content into a help request on GitHub (or where ever).
>
> If you are curious to see what's collected, the log is here:
> /home/m4madmin/ECS-CommunityEdition/install.log
>
> It is perfectly fine to remove this log file at any time.
>
> Onward to bootstrapping. This can take anywhere between five
> minutes to a few hours depending on many factors, the most
> important being the speed of your Internet connection.
> The ECS software docker image is around 1.5GiB.
> Bootstrapping requires about 10 minutes under KVM on a
> Xeon E5 with a 250Mbps Internet connection.
>
> This script installs all packages that are both required for
> the deployment and that we think will be helpful to you when
> managing and operating your environment.
>
> We're going to start working with Docker now. If you elected
> to build your own ecs-install image rather than pull one from
> the EMC Dockerhub repo, it will add some time to your initial
> bootstrap.
>
> We are now pulling the emccorp/ecs-software-3.4.0 image.
> This can take quite a long time over a slow Internet link or
> if the backing block storage system is slower than usual.
>
> All done bootstrapping your install node.
>
> To continue (after reboot if needed):
> $ cd /home/m4madmin/ECS-CommunityEdition
> If you have a deploy.yml ready to go (and did not use -c flag):
> $ sudo cp deploy.yml /opt/emc/ecs-install/
> If not, check out the docs/design and examples directory for references.
> Once you have a deploy.yml, you can start the deployment
> by running:
>
> [WITH Internet access]
> $ step1
> [Wait for deployment to complete, then run:]
> $ step2
>
> [WITHOUT Internet access]
> $ island-step1
> [Migrate your install node into the isolated environment and run:]
> $ island-step2
> [Wait for deployment to complete, then run:]
> $ island-step3
>
[m4madmin@luna ECS-CommunityEdition]$ step1
ecs-install> Initializing data container, one moment ..DEPRECATION: Python 2.7 will reach the end of its life on January 1st, 2020. Please upgrade your Python as Python 2.7 won't be maintained after that date. A future version of pip will drop support for Python 2.7. More details about Python 2 support in pip, can be found at https://pip.pypa.io/en/latest/development/release-process/#python-2-support
ERROR: Command errored out with exit status 1:
command: /usr/bin/python2 /usr/lib/python2.7/site-packages/pip install --ignore-installed --no-user --prefix /tmp/pip-build-env-WlVl06/overlay --no-warn-script-location --no-binary :none: --only-binary :none: -i https://pypi.org/simple -- 'setuptools>=40.6.0' wheel 'cffi>=1.8,!=1.11.3; platform_python_implementation != '"'"'PyPy'"'"''
cwd: None
Complete output (69 lines):
DEPRECATION: Python 2.7 will reach the end of its life on January 1st, 2020. Please upgrade your Python as Python 2.7 won't be maintained after that date. A future version of pip will drop support for Python 2.7. More details about Python 2 support in pip, can be found at https://pip.pypa.io/en/latest/development/release-process/#python-2-support
Collecting setuptools>=40.6.0
Downloading https://files.pythonhosted.org/packages/f9/d3/955738b20d3832dfa3cd3d9b07e29a8162edb480bf988332f5e6e48ca444/setuptools-44.0.0-py2.py3-none-any.whl (583kB)
Collecting wheel
Downloading https://files.pythonhosted.org/packages/00/83/b4a77d044e78ad1a45610eb88f745be2fd2c6d658f9798a15e384b7d57c9/wheel-0.33.6-py2.py3-none-any.whl
Collecting cffi!=1.11.3,>=1.8
Downloading https://files.pythonhosted.org/packages/2d/bf/960e5a422db3ac1a5e612cb35ca436c3fc985ed4b7ed13a1b4879006f450/cffi-1.13.2.tar.gz (460kB)
Collecting pycparser
Downloading https://files.pythonhosted.org/packages/68/9e/49196946aee219aead1290e00d1e7fdeab8567783e83e1b9ab5585e6206a/pycparser-2.19.tar.gz (158kB)
ERROR: pyopenssl 19.1.0 has requirement cryptography>=2.8, but you'll have cryptography 2.1.4 which is incompatible.
Installing collected packages: setuptools, wheel, pycparser, cffi
Running setup.py install for pycparser: started
Running setup.py install for pycparser: finished with status 'done'
Running setup.py install for cffi: started
Running setup.py install for cffi: finished with status 'error'
ERROR: Command errored out with exit status 1:
command: /usr/bin/python2 -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-7iIsEW/cffi/setup.py'"'"'; __file__='"'"'/tmp/pip-install-7iIsEW/cffi/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' install --record /tmp/pip-record-aHDJhb/install-record.txt --single-version-externally-managed --prefix /tmp/pip-build-env-WlVl06/overlay --compile
cwd: /tmp/pip-install-7iIsEW/cffi/
Complete output (48 lines):
unable to execute 'gcc': No such file or directory
unable to execute 'gcc': No such file or directory
No working compiler found, or bogus compiler options passed to
the compiler from Python's standard "distutils" module. See
the error messages above. Likely, the problem is not related
to CFFI but generic to the setup.py of any Python package that
tries to compile C code. (Hints: on OS/X 10.8, for errors about
-mno-fused-madd see http://stackoverflow.com/questions/22313407/
Otherwise, see https://wiki.python.org/moin/CompLangPython or
the IRC channel #python on irc.freenode.net.)
Trying to continue anyway. If you are trying to install CFFI from
a build done in a different context, you can ignore this warning.
running install
running build
running build_py
creating build
creating build/lib.linux-x86_64-2.7
creating build/lib.linux-x86_64-2.7/cffi
copying cffi/vengine_cpy.py -> build/lib.linux-x86_64-2.7/cffi
copying cffi/model.py -> build/lib.linux-x86_64-2.7/cffi
copying cffi/commontypes.py -> build/lib.linux-x86_64-2.7/cffi
copying cffi/error.py -> build/lib.linux-x86_64-2.7/cffi
copying cffi/backend_ctypes.py -> build/lib.linux-x86_64-2.7/cffi
copying cffi/cparser.py -> build/lib.linux-x86_64-2.7/cffi
copying cffi/ffiplatform.py -> build/lib.linux-x86_64-2.7/cffi
copying cffi/pkgconfig.py -> build/lib.linux-x86_64-2.7/cffi
copying cffi/vengine_gen.py -> build/lib.linux-x86_64-2.7/cffi
copying cffi/api.py -> build/lib.linux-x86_64-2.7/cffi
copying cffi/lock.py -> build/lib.linux-x86_64-2.7/cffi
copying cffi/__init__.py -> build/lib.linux-x86_64-2.7/cffi
copying cffi/setuptools_ext.py -> build/lib.linux-x86_64-2.7/cffi
copying cffi/cffi_opcode.py -> build/lib.linux-x86_64-2.7/cffi
copying cffi/verifier.py -> build/lib.linux-x86_64-2.7/cffi
copying cffi/recompiler.py -> build/lib.linux-x86_64-2.7/cffi
copying cffi/_cffi_include.h -> build/lib.linux-x86_64-2.7/cffi
copying cffi/parse_c_type.h -> build/lib.linux-x86_64-2.7/cffi
copying cffi/_embedding.h -> build/lib.linux-x86_64-2.7/cffi
copying cffi/_cffi_errors.h -> build/lib.linux-x86_64-2.7/cffi
running build_ext
building '_cffi_backend' extension
creating build/temp.linux-x86_64-2.7
creating build/temp.linux-x86_64-2.7/c
gcc -fno-strict-aliasing -Os -fomit-frame-pointer -g -DNDEBUG -Os -fomit-frame-pointer -g -DTHREAD_STACK_SIZE=0x100000 -fPIC -I/usr/include/ffi -I/usr/include/libffi -I/usr/include/python2.7 -c c/_cffi_backend.c -o build/temp.linux-x86_64-2.7/c/_cffi_backend.o
unable to execute 'gcc': No such file or directory
error: command 'gcc' failed with exit status 1
----------------------------------------
ERROR: Command errored out with exit status 1: /usr/bin/python2 -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-7iIsEW/cffi/setup.py'"'"'; __file__='"'"'/tmp/pip-install-7iIsEW/cffi/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' install --record /tmp/pip-record-aHDJhb/install-record.txt --single-version-externally-managed --prefix /tmp/pip-build-env-WlVl06/overlay --compile Check the logs for full command output.
----------------------------------------
ERROR: Command errored out with exit status 1: /usr/bin/python2 /usr/lib/python2.7/site-packages/pip install --ignore-installed --no-user --prefix /tmp/pip-build-env-WlVl06/overlay --no-warn-script-location --no-binary :none: --only-binary :none: -i https://pypi.org/simple -- 'setuptools>=40.6.0' wheel 'cffi>=1.8,!=1.11.3; platform_python_implementation != '"'"'PyPy'"'"'' Check the logs for full command output.
ecs-install> ui.tgz failed to build.
. OK
ecs-install> Applying and validating deploy.yml...
/usr/local/bin/entrypoint.sh: line 167: ecsdeploy: not found
/usr/local/bin/entrypoint.sh: line 232: ecsdeploy: not found
Any help greatly appreciated!
Notifies: @captntuttle
I think you dont have GCC installed.
Hi salieri11, I did install gcc on my installation node when I first hit this error, but I think the error is actually being generated from within the 'ecs-install-data' Docker container (which is an instance of the 'emccorp/ecs-install:latest' image, which the bootstrap process has built). GCC won't be part of the minimal Alpine Linux image the ecs-install is based on, so the error makes sense from that perspective.
My suspicion is that the source of the problem is the 'pip' process running inside the container that's detecting an incompatibility:
ERROR: pyopenssl 19.1.0 has requirement cryptography>=2.8, but you'll have cryptography 2.1.4 which is incompatible.
and is then trying to build the required component from source, which is failing due to the lack of GCC. If this is the case, I think we either need to get the bootstrap process to build an image that has the correct dependencies container within it (so no need for GCC), or add GCC to the base Alpine Linux image as part of the build process.
Ok, so the solution seems to be to edit the following file:
ECS-CommunityEdition/ui/resources/docker/ecs-install-requirements.txt
and change the following line:
cryptography>=2.1.4
to:
cryptography>=2.8
After doing this and redoing the bootstrap stage to rebuild the image, 'step1' starts correctly:
step1
ecs-install> Initializing data container, one moment ..DEPRECATION: Python 2.7 will reach the end of its life on January 1st, 2020. Please upgrade your Python as Python 2.7 won't be maintained after that date. A future version of pip will drop support for Python 2.7. More details about Python 2 support in pip, can be found at https://pip.pypa.io/en/latest/development/release-process/#python-2-support
. OK
ecs-install> Applying and validating deploy.yml...
PLAY [Installer | Setup access between ecs-install and install node] *****************************************************************************************************************************************************************
TASK [Gathering Facts] ***************************************************************************************************************************************************************************************************************
... and so on...
This issue seems to have been caused by a backward-incompatible change to pyOpenSSL in 19.1.0 (2019-11-18) - see https://pypi.org/project/pyOpenSSL/
19.1.0 (2019-11-18) Backward-incompatible changes:
- Removed deprecated ContextType, ConnectionType, PKeyType, X509NameType, X509ReqType, X509Type, X509StoreType, CRLType, PKCS7Type, PKCS12Type, and NetscapeSPKIType aliases. Use the classes without the Type suffix instead. #814
- The minimum cryptography version is now 2.8 due to issues on macOS with a transitive dependency. #875
@MookThompson Sorry for the late response here. I'll be sure to merge this change in. There were quite a few dependency issues in the new Alpine build that I had to work through.
