cloudgoat
cloudgoat copied to clipboard
Published
20 hours ago •
RhinoSecurityLabs
RhinoSecurityLabs
Fix container build on non-arm64 hosts.
This changes the Dockerfile to be based on the upstream docker container rather then pulling the executable down ourselves. Before we where accidentally fetching the arm64 version, which of course won't work on amd64 hosts.
I'm not positive, but I believe doing it this way is platform independent, if someone has a arm64 mac and can test this that would be appreciated. In any case this resolves the issue running the container on my amd host.
Resolves #122
Looks good. docker build succeeded except for this error message in step 8:
Step 8/12 : RUN pip3 install -r ./requirements.txt
---> Running in f6acbc30cd7c
Collecting argcomplete==1.10.0
Downloading argcomplete-1.10.0-py2.py3-none-any.whl (31 kB)
Collecting PyYAML==5.4
Downloading PyYAML-5.4.tar.gz (174 kB)
Installing build dependencies: started
Installing build dependencies: finished with status 'done'
Getting requirements to build wheel: started
Getting requirements to build wheel: finished with status 'done'
Preparing wheel metadata: started
Preparing wheel metadata: finished with status 'done'
Collecting boto3==1.18.1
Downloading boto3-1.18.1-py3-none-any.whl (131 kB)
Collecting requests==2.26.0
Downloading requests-2.26.0-py2.py3-none-any.whl (62 kB)
Collecting botocore<1.22.0,>=1.21.1
Downloading botocore-1.21.65-py3-none-any.whl (8.0 MB)
Requirement already satisfied: s3transfer<0.6.0,>=0.5.0 in /usr/lib/python3.8/site-packages (from boto3==1.18.1->-r ./requirements.txt (line 7)) (0.5.1)
Requirement already satisfied: jmespath<1.0.0,>=0.7.1 in /usr/lib/python3.8/site-packages (from boto3==1.18.1->-r ./requirements.txt (line 7)) (0.10.0)
Requirement already satisfied: idna<4,>=2.5 in /usr/lib/python3.8/site-packages (from requests==2.26.0->-r ./requirements.txt (line 8)) (3.1)
Collecting certifi>=2017.4.17
Downloading certifi-2021.10.8-py2.py3-none-any.whl (149 kB)
Collecting charset-normalizer~=2.0.0
Downloading charset_normalizer-2.0.12-py3-none-any.whl (39 kB)
Requirement already satisfied: urllib3<1.27,>=1.21.1 in /usr/lib/python3.8/site-packages (from requests==2.26.0->-r ./requirements.txt (line 8)) (1.26.2)
Requirement already satisfied: python-dateutil<3.0.0,>=2.1 in /usr/lib/python3.8/site-packages (from botocore<1.22.0,>=1.21.1->boto3==1.18.1->-r ./requirements.txt (line 7)) (2.8.2)
Requirement already satisfied: six>=1.5 in /usr/lib/python3.8/site-packages (from python-dateutil<3.0.0,>=2.1->botocore<1.22.0,>=1.21.1->boto3==1.18.1->-r ./requirements.txt (line 7)) (1.15.0)
Building wheels for collected packages: PyYAML
Building wheel for PyYAML (PEP 517): started
Building wheel for PyYAML (PEP 517): finished with status 'done'
Created wheel for PyYAML: filename=PyYAML-5.4-cp38-cp38-linux_x86_64.whl size=45524 sha256=2df0a8960de35301f5b0e795c0e97a291ddbae77733afdd01cc8cbe6fef8ccec
Stored in directory: /root/.cache/pip/wheels/89/b4/5f/cb055d6b1decb1050d7a3620abf3bca515f16d28e8c42ae23e
Successfully built PyYAML
Installing collected packages: botocore, charset-normalizer, certifi, requests, PyYAML, boto3, argcomplete
Attempting uninstall: botocore
Found existing installation: botocore 1.24.3
Uninstalling botocore-1.24.3:
Successfully uninstalled botocore-1.24.3
Attempting uninstall: requests
Found existing installation: requests 2.25.1
Uninstalling requests-2.25.1:
Successfully uninstalled requests-2.25.1
Attempting uninstall: PyYAML
Found existing installation: PyYAML 5.4.1
Uninstalling PyYAML-5.4.1:
Successfully uninstalled PyYAML-5.4.1
ERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts.
awscli 1.22.58 requires botocore==1.24.3, but you have botocore 1.21.65 which is incompatible.
Successfully installed PyYAML-5.4 argcomplete-1.10.0 boto3-1.18.1 botocore-1.21.65 certifi-2021.10.8 charset-normalizer-2.0.12 requests-2.26.0
Removing intermediate container f6acbc30cd7c
terraform executes correctly in the target
@eddydee123 odd, I can't seem to reproduce this. Can you share what OS/CPU Architecture you're running on?
I'm running on an AWS EC2 Amazon Linux2 instance: Linux ip-10-0-0-36.ec2.internal 5.10.96-90.460.amzn2.x86_64 #1 SMP Fri Feb 4 17:12:04 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux
In case it helps, here's the full output:
Sending build context to Docker daemon 16.3MB
Step 1/12 : FROM hashicorp/terraform:0.15.1
---> 8caf8068d384
Step 2/12 : LABEL maintainer="Rhino Assessment Team <[email protected]>"
---> Running in 8376030bdd82
Removing intermediate container 8376030bdd82
---> 81a13d788736
Step 3/12 : LABEL cloudgoat.version="2.0.0"
---> Running in bc1b4352d661
Removing intermediate container bc1b4352d661
---> 97cbaa4d0097
Step 4/12 : RUN apk add --no-cache --update bash bash-completion docker-bash-completion openssh curl
---> Running in 0f520229c6f9
fetch https://dl-cdn.alpinelinux.org/alpine/v3.13/main/x86_64/APKINDEX.tar.gz
fetch https://dl-cdn.alpinelinux.org/alpine/v3.13/community/x86_64/APKINDEX.tar.gz
(1/7) Installing readline (8.1.0-r0)
(2/7) Installing bash (5.1.0-r0)
Executing bash-5.1.0-r0.post-install
(3/7) Installing pkgconf (1.7.3-r0)
(4/7) Installing bash-completion (2.11-r2)
(5/7) Installing curl (7.79.1-r0)
(6/7) Installing docker-bash-completion (20.10.3-r1)
(7/7) Installing git-bash-completion (2.30.2-r0)
Executing busybox-1.32.1-r2.trigger
OK: 29 MiB in 37 packages
Removing intermediate container 0f520229c6f9
---> 430fbb9caaf9
Step 5/12 : RUN apk update && apk add python3 py3-pip && pip3 install awscli --upgrade
---> Running in 0be59dda58de
fetch https://dl-cdn.alpinelinux.org/alpine/v3.13/main/x86_64/APKINDEX.tar.gz
fetch https://dl-cdn.alpinelinux.org/alpine/v3.13/community/x86_64/APKINDEX.tar.gz
v3.13.7-72-g115793f18b [https://dl-cdn.alpinelinux.org/alpine/v3.13/main]
v3.13.7-71-geae8cd3ed9 [https://dl-cdn.alpinelinux.org/alpine/v3.13/community]
OK: 13915 distinct packages available
(1/32) Installing libbz2 (1.0.8-r1)
(2/32) Installing libffi (3.3-r2)
(3/32) Installing gdbm (1.19-r0)
(4/32) Installing xz-libs (5.2.5-r0)
(5/32) Installing sqlite-libs (3.34.1-r0)
(6/32) Installing python3 (3.8.10-r0)
(7/32) Installing py3-appdirs (1.4.4-r1)
(8/32) Installing py3-ordered-set (4.0.2-r0)
(9/32) Installing py3-parsing (2.4.7-r1)
(10/32) Installing py3-six (1.15.0-r0)
(11/32) Installing py3-packaging (20.9-r0)
(12/32) Installing py3-setuptools (51.3.3-r0)
(13/32) Installing py3-chardet (4.0.0-r0)
(14/32) Installing py3-idna (3.1-r0)
(15/32) Installing py3-urllib3 (1.26.2-r1)
(16/32) Installing py3-requests (2.25.1-r1)
(17/32) Installing py3-msgpack (1.0.2-r0)
(18/32) Installing py3-lockfile (0.12.2-r3)
(19/32) Installing py3-cachecontrol (0.12.6-r0)
(20/32) Installing py3-colorama (0.4.4-r0)
(21/32) Installing py3-contextlib2 (0.6.0-r0)
(22/32) Installing py3-distlib (0.3.1-r1)
(23/32) Installing py3-distro (1.5.0-r1)
(24/32) Installing py3-webencodings (0.5.1-r3)
(25/32) Installing py3-html5lib (1.1-r0)
(26/32) Installing py3-pytoml (0.1.21-r0)
(27/32) Installing py3-pep517 (0.9.1-r0)
(28/32) Installing py3-progress (1.5-r0)
(29/32) Installing py3-retrying (1.3.3-r0)
(30/32) Installing py3-toml (0.10.2-r0)
(31/32) Installing py3-pip (20.3.4-r0)
(32/32) Installing py3-pip-bash-completion (20.3.4-r0)
Executing busybox-1.32.1-r2.trigger
OK: 97 MiB in 69 packages
Collecting awscli
Downloading awscli-1.22.69-py3-none-any.whl (3.8 MB)
Collecting docutils<0.16,>=0.10
Downloading docutils-0.15.2-py3-none-any.whl (547 kB)
Collecting botocore==1.24.14
Downloading botocore-1.24.14-py3-none-any.whl (8.6 MB)
Collecting s3transfer<0.6.0,>=0.5.0
Downloading s3transfer-0.5.2-py3-none-any.whl (79 kB)
Collecting PyYAML<5.5,>=3.10
Downloading PyYAML-5.4.1.tar.gz (175 kB)
Installing build dependencies: started
Installing build dependencies: finished with status 'done'
Getting requirements to build wheel: started
Getting requirements to build wheel: finished with status 'done'
Preparing wheel metadata: started
Preparing wheel metadata: finished with status 'done'
Collecting colorama<0.4.4,>=0.2.5
Downloading colorama-0.4.3-py2.py3-none-any.whl (15 kB)
Collecting rsa<4.8,>=3.1.2
Downloading rsa-4.7.2-py3-none-any.whl (34 kB)
Requirement already satisfied: urllib3<1.27,>=1.25.4 in /usr/lib/python3.8/site-packages (from botocore==1.24.14->awscli) (1.26.2)
Collecting jmespath<1.0.0,>=0.7.1
Downloading jmespath-0.10.0-py2.py3-none-any.whl (24 kB)
Collecting python-dateutil<3.0.0,>=2.1
Downloading python_dateutil-2.8.2-py2.py3-none-any.whl (247 kB)
Requirement already satisfied: six>=1.5 in /usr/lib/python3.8/site-packages (from python-dateutil<3.0.0,>=2.1->botocore==1.24.14->awscli) (1.15.0)
Collecting pyasn1>=0.1.3
Downloading pyasn1-0.4.8-py2.py3-none-any.whl (77 kB)
Building wheels for collected packages: PyYAML
Building wheel for PyYAML (PEP 517): started
Building wheel for PyYAML (PEP 517): finished with status 'done'
Created wheel for PyYAML: filename=PyYAML-5.4.1-cp38-cp38-linux_x86_64.whl size=45656 sha256=a5c1a79bb284b792b7d02b4560dd30d1d0b00237038afc0fb75e5bc46adac334
Stored in directory: /root/.cache/pip/wheels/dd/c5/1d/5d7436173d3efd4a14dcb510eb0b29525ecb6b0e41489e716e
Successfully built PyYAML
Installing collected packages: python-dateutil, jmespath, pyasn1, botocore, s3transfer, rsa, PyYAML, docutils, colorama, awscli
Attempting uninstall: colorama
Found existing installation: colorama 0.4.4
Uninstalling colorama-0.4.4:
Successfully uninstalled colorama-0.4.4
Successfully installed PyYAML-5.4.1 awscli-1.22.69 botocore-1.24.14 colorama-0.4.3 docutils-0.15.2 jmespath-0.10.0 pyasn1-0.4.8 python-dateutil-2.8.2 rsa-4.7.2 s3transfer-0.5.2
Removing intermediate container 0be59dda58de
---> 47451309bd57
Step 6/12 : WORKDIR /usr/src/cloudgoat/core/python
---> Running in 81a5ef01d105
Removing intermediate container 81a5ef01d105
---> 1c9198ba3f57
Step 7/12 : COPY ./core/python/requirements.txt ./
---> a258b0937b72
Step 8/12 : RUN pip3 install -r ./requirements.txt
---> Running in 0e3f4b890f7c
Collecting argcomplete==1.10.0
Downloading argcomplete-1.10.0-py2.py3-none-any.whl (31 kB)
Collecting PyYAML==5.4
Downloading PyYAML-5.4.tar.gz (174 kB)
Installing build dependencies: started
Installing build dependencies: finished with status 'done'
Getting requirements to build wheel: started
Getting requirements to build wheel: finished with status 'done'
Preparing wheel metadata: started
Preparing wheel metadata: finished with status 'done'
Collecting boto3==1.18.1
Downloading boto3-1.18.1-py3-none-any.whl (131 kB)
Collecting requests==2.26.0
Downloading requests-2.26.0-py2.py3-none-any.whl (62 kB)
Collecting botocore<1.22.0,>=1.21.1
Downloading botocore-1.21.65-py3-none-any.whl (8.0 MB)
Requirement already satisfied: jmespath<1.0.0,>=0.7.1 in /usr/lib/python3.8/site-packages (from boto3==1.18.1->-r ./requirements.txt (line 7)) (0.10.0)
Requirement already satisfied: s3transfer<0.6.0,>=0.5.0 in /usr/lib/python3.8/site-packages (from boto3==1.18.1->-r ./requirements.txt (line 7)) (0.5.2)
Collecting certifi>=2017.4.17
Downloading certifi-2021.10.8-py2.py3-none-any.whl (149 kB)
Collecting charset-normalizer~=2.0.0
Downloading charset_normalizer-2.0.12-py3-none-any.whl (39 kB)
Requirement already satisfied: urllib3<1.27,>=1.21.1 in /usr/lib/python3.8/site-packages (from requests==2.26.0->-r ./requirements.txt (line 8)) (1.26.2)
Requirement already satisfied: idna<4,>=2.5 in /usr/lib/python3.8/site-packages (from requests==2.26.0->-r ./requirements.txt (line 8)) (3.1)
Requirement already satisfied: python-dateutil<3.0.0,>=2.1 in /usr/lib/python3.8/site-packages (from botocore<1.22.0,>=1.21.1->boto3==1.18.1->-r ./requirements.txt (line 7)) (2.8.2)
Requirement already satisfied: six>=1.5 in /usr/lib/python3.8/site-packages (from python-dateutil<3.0.0,>=2.1->botocore<1.22.0,>=1.21.1->boto3==1.18.1->-r ./requirements.txt (line 7)) (1.15.0)
Building wheels for collected packages: PyYAML
Building wheel for PyYAML (PEP 517): started
Building wheel for PyYAML (PEP 517): finished with status 'done'
Created wheel for PyYAML: filename=PyYAML-5.4-cp38-cp38-linux_x86_64.whl size=45524 sha256=e1461b6309d44622c155f0eef95864d7d0efda135e44d342de022dc699b7b7ad
Stored in directory: /root/.cache/pip/wheels/89/b4/5f/cb055d6b1decb1050d7a3620abf3bca515f16d28e8c42ae23e
Successfully built PyYAML
Installing collected packages: botocore, charset-normalizer, certifi, requests, PyYAML, boto3, argcomplete
Attempting uninstall: botocore
Found existing installation: botocore 1.24.14
Uninstalling botocore-1.24.14:
Successfully uninstalled botocore-1.24.14
Attempting uninstall: requests
Found existing installation: requests 2.25.1
Uninstalling requests-2.25.1:
Successfully uninstalled requests-2.25.1
Attempting uninstall: PyYAML
Found existing installation: PyYAML 5.4.1
Uninstalling PyYAML-5.4.1:
Successfully uninstalled PyYAML-5.4.1
ERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts.
awscli 1.22.69 requires botocore==1.24.14, but you have botocore 1.21.65 which is incompatible.
Successfully installed PyYAML-5.4 argcomplete-1.10.0 boto3-1.18.1 botocore-1.21.65 certifi-2021.10.8 charset-normalizer-2.0.12 requests-2.26.0
Removing intermediate container 0e3f4b890f7c
---> 12a69c26730f
Step 9/12 : WORKDIR /usr/src/cloudgoat/
---> Running in 2cb649e47bcf
Removing intermediate container 2cb649e47bcf
---> 3f027e22226f
Step 10/12 : COPY ./ ./
---> 8ca4cd00d482
Step 11/12 : ENTRYPOINT ["/bin/bash", "-c"]
---> Running in d8eda88abeae
Removing intermediate container d8eda88abeae
---> 68cc0ff4d047
Step 12/12 : CMD ["bash", "-l"]
---> Running in 926a6e4d686e
Removing intermediate container 926a6e4d686e
---> e6eef11e241e
Successfully built e6eef11e241e
[ec2-user@ip-10-0-0-36 cloudgoat]$ pip3 --version
pip 20.2.2 from /usr/lib/python3.7/site-packages/pip (python 3.7)
I checked on the docker hub page for terraform, that the image that is being used, does not offer arm64 as a os arch. hence the image only works on x86_64/amd64. So the question is how we can make this platform specific. I thought that in terraform, you can use multi-arch since 2019... https://www.docker.com/blog/multi-arch-images/
It seems, it would be better to include the arch and use this as a prefix in the Dockerfile: ARG ARCH= FROM ${ARCH}\debian:buster-slim
