awx icon indicating copy to clipboard operation
awx copied to clipboard
verified publisher icon ansible

Doesn't build without VERSION file in the repo root

Open altbrace opened this issue 1 year ago • 4 comments

Please confirm the following

  • [X] I agree to follow this project's code of conduct.
  • [X] I have checked the current issues for duplicates.
  • [X] I understand that AWX is open source software provided for free and that I might not receive a timely response.
  • [X] I am NOT reporting a (potential) security vulnerability. (These should be emailed to [email protected] instead.)

Bug Summary

I'm building an AWX Execution Environment with ansible-builder. I see that while installing ansible-runner, it needs an AWX tarball as a dependency. It builds it using pyproject.toml, but to determine the version, it need the VERSION file in the repo root. I've forked the repo and tried it out - the problem seems to be fixed.

No VERSION file: no_version_build.log

version: 3

build_arg_defaults:
  ANSIBLE_GALAXY_CLI_COLLECTION_OPTS: '--pre'

dependencies:
  ansible_core:
    package_pip: ansible-core==2.15.12
  ansible_runner:
    package_pip: ansible-runner==2.4.0
  galaxy: requirements.yml
  python: requirements.txt
  system: bindep.txt
  python_interpreter:
    package_system: "python3"
    python_path: "/usr/bin/python3.9"

images:
  base_image:
    name: docker.io/redhat/ubi9:latest

additional_build_files:
  - src: awx-24.6.1.tar.gz
    dest: dist
additional_build_steps:
  prepend_base:
     - RUN yum remove -y xmlsec1-1.2.29-9 xmlsec1-openssl-1.2.29-9
     - RUN yum install -y --allowerasing pkg-config https://repo.almalinux.org/almalinux/9/CRB/x86_64/os/Packages/libtool-ltdl-devel-2.4.6-45.el9.x86_64.rpm https://repo.almalinux.org/almalinux/8/AppStream/x86_64/os/Packages/xmlsec1-openssl-1.2.25-4.el8.x86_64.rpm https://repo.almalinux.org/almalinux/8/AppStream/x86_64/os/Packages/xmlsec1-1.2.25-4.el8.x86_64.rpm https://vault.centos.org/centos/8/PowerTools/x86_64/os/Packages/xmlsec1-devel-1.2.25-4.el8.x86_64.rpm
     - COPY _build/dist/awx-24.6.1.tar.gz /tmp/src/dist/awx.tar.gz
  prepend_builder:
    - ENV PKG_CONFIG_PATH=/usr/local/lib/x86_64-linux-gnu/pkgconfig:/usr/local/lib/pkgconfig:/usr/local/share/pkgconfig:/usr/lib/x86_64-linux-gnu/pkgconfig:/usr/lib/pkgconfig:/usr/share/pkgconfig
    - RUN /usr/bin/python3.9 -m pip install --upgrade pip
    - RUN yum remove -y python3-requests

With VERSION file: with_version_build.log

version: 3

build_arg_defaults:
  ANSIBLE_GALAXY_CLI_COLLECTION_OPTS: '--pre'

dependencies:
  ansible_core:
    package_pip: ansible-core==2.15.12
  ansible_runner:
    package_pip: ansible-runner==2.4.0
  galaxy: requirements.yml
  python: requirements.txt
  system: bindep.txt
  python_interpreter:
    package_system: "python3"
    python_path: "/usr/bin/python3.9"

images:
  base_image:
    name: docker.io/redhat/ubi9:latest

additional_build_files:
  - src: awx-24.6.2.tar.gz
    dest: dist
additional_build_steps:
  prepend_base:
     - RUN yum remove -y xmlsec1-1.2.29-9 xmlsec1-openssl-1.2.29-9
     - RUN yum install -y --allowerasing pkg-config https://repo.almalinux.org/almalinux/9/CRB/x86_64/os/Packages/libtool-ltdl-devel-2.4.6-45.el9.x86_64.rpm https://repo.almalinux.org/almalinux/8/AppStream/x86_64/os/Packages/xmlsec1-openssl-1.2.25-4.el8.x86_64.rpm https://repo.almalinux.org/almalinux/8/AppStream/x86_64/os/Packages/xmlsec1-1.2.25-4.el8.x86_64.rpm https://vault.centos.org/centos/8/PowerTools/x86_64/os/Packages/xmlsec1-devel-1.2.25-4.el8.x86_64.rpm
     - COPY _build/dist/awx-24.6.2.tar.gz /tmp/src/dist/awx.tar.gz
  prepend_builder:
    - ENV PKG_CONFIG_PATH=/usr/local/lib/x86_64-linux-gnu/pkgconfig:/usr/local/lib/pkgconfig:/usr/local/share/pkgconfig:/usr/lib/x86_64-linux-gnu/pkgconfig:/usr/lib/pkgconfig:/usr/share/pkgconfig
    - RUN /usr/bin/python3.9 -m pip install --upgrade pip
    - RUN yum remove -y python3-requests

VERSION file contents: 24.6.1

AWX version

24.6.1

Select the relevant components

  • [ ] UI
  • [ ] UI (tech preview)
  • [ ] API
  • [ ] Docs
  • [ ] Collection
  • [x] CLI
  • [ ] Other

Installation method

kubernetes

Modifications

no

Ansible version

2.15.2

Operating system

linux

Web browser

No response

Steps to reproduce

  1. Install ansible-builder using pip install ansible-builder
  2. Create a directory for the build and cd to it: cd awx-ee-test
  3. Download and move AWX 24.6.1 tarball to the root
  4. Create bindep.txt file and fill it like this:
git                          [platform:rpm]
openldap-devel     [platform:rpm]
openssl-devel        [platform:rpm]
bzip2-devel            [platform:rpm]
libffi-devel              [platform:rpm]
sqlite-devel            [platform:rpm]
python3.12             [platform:rpm]
make                       [platform:rpm]
gcc                          [platform:rpm]
python3-devel        [platform:rpm]
pkg-config              [platform:rpm]
libxml2                    [platform:rpm]
libxml2-devel          [platform:rpm]
libxslt                      [platform:rpm]
zlib                          [platform:rpm]
libxslt-devel            [platform:rpm]
  1. Create requirements.yml file and fill it like this:
---
collections:
  - name: ansible.utils
    version: 5.0.0
  - name: awx.awx
    version: 24.6.1
  - name: community.general
    version: 9.2.0
  1. Create requirements.txt file and fill it like this:
adal==1.2.7
aiohttp==3.8.3
aioredis==1.3.1
aiosignal==1.3.1
ansible-runner==2.4.0
ansiconv==1.0.0
asciichartpy==1.5.25
asgiref==3.6.0
asn1==2.6.0
async-timeout==4.0.2
asyncpg==0.27.0
attrs==22.1.0
autobahn==22.7.1
autocommand==2.2.2
Automat==22.10.0
awx @ file:///tmp/src/dist/awx.tar.gz
azure-common==1.1.28
azure-core==1.26.1
azure-keyvault==1.1.0
azure-nspkg==3.0.2
boto3==1.26.102
botocore==1.29.102
cachetools==5.2.0
certifi @ git+https://github.com/ansible/system-certifi.git@5aa52ab91f9d579bfe52b5acf30ca799f1a563d9
cffi==1.15.1
channels==3.0.5
channels-redis==3.4.1
charset-normalizer==2.1.1
click==8.1.3
constantly==15.1.0
cryptography==41.0.3
Cython==0.29.32
daphne==3.0.2
defusedxml==0.7.1
Deprecated==1.2.13
distro==1.8.0
Django==4.2.6
django-auth-ldap==4.1.0
django-cors-headers==3.13.0
django-crum==0.7.9
django-extensions==3.2.1
django-guid==3.2.1
django-oauth-toolkit==1.7.1
django-pglocks==1.0.4
django-polymorphic==3.1.0
django-radius @ git+https://github.com/ansible/django-radius.git@0f514610eb28648f5b07930777f85ed5f4a62bda
django-solo==2.0.0
django-split-settings==1.0.0
djangorestframework==3.14.0
djangorestframework-yaml==2.0.0
docutils==0.19
ecdsa==0.18.0
enum-compat==0.0.3
filelock==3.8.0
frozenlist==1.3.3
gitdb==4.0.10
GitPython==3.1.32
google-auth==2.14.1
hiredis==2.0.0
hyperlink==21.0.0
idna==3.4
importlib-metadata==4.6.4
incremental==22.10.0
inflect==6.0.2
irc==20.1.0
isodate==0.6.1
jaraco.classes==3.2.3
jaraco.collections==3.8.0
jaraco.context==4.2.0
jaraco.functools==3.5.2
jaraco.logging==3.1.2
jaraco.stream==3.0.3
jaraco.text==3.11.0
Jinja2==3.1.2
jmespath==1.0.1
JSON-log-formatter==0.5.1
jsonschema==4.17.3
jwcrypto==1.4.2
kubernetes==25.3.0
lockfile==0.12.2
lxml==4.9.1
Markdown==3.4.1
MarkupSafe==2.1.1
more-itertools==9.0.0
msgpack==1.0.4
msrest==0.7.1
msrestazure==0.6.4
multidict==6.0.2
netaddr>=0.10.1
oauthlib==3.2.2
openshift==0.13.1
packaging==21.3
pbr==5.11.0
pexpect==4.7.0
pkgconfig==1.5.5
prometheus-client==0.15.0
psutil==5.9.4
psycopg==3.1.9
ptyprocess==0.7.0
pyasn1==0.4.8
pyasn1-modules==0.2.8
pycparser==2.21
pydantic==1.10.2
pygerduty==0.38.3
PyJWT==2.6.0
pyOpenSSL==23.2.0
pyparsing==2.4.6
pyrad==2.4
pyrsistent==0.19.2
python-daemon==3.0.1
python-dateutil==2.8.2
python-dsv-sdk==1.0.4
python-jose==3.3.0
python-ldap==3.4.3
python-string-utils==1.0.0
python-tss-sdk==1.2.1
python3-openid==3.2.0
python3-saml @ git+https://github.com/ansible/python3-saml.git@f90824c4910e36c5a89dd295271be26691204ba3
pytz==2022.6
PyYAML==6.0.1
receptorctl==1.4.2
redis==4.3.5
requests==2.28.1
requests-oauthlib==1.3.1
s3transfer==0.6.0
semantic-version==2.10.0
service-identity==21.1.0
setuptools-rust==1.5.2
setuptools-scm==7.0.5
six==1.16.0
slack-sdk==3.19.4
smmap==5.0.0
social-auth-app-django==5.0.0
social-auth-core==4.3.0
sqlparse==0.4.4
tacacs-plus==1.0
tempora==5.1.0
tomli==2.0.1
twilio==7.15.3
Twisted==22.10.0
txaio==22.2.1
typing_extensions==4.4.0
urllib3==1.26.13
uWSGI==2.0.21
uwsgitop==0.11
websocket-client==1.4.2
wrapt==1.15.0
xmlsec==1.3.13
yarl==1.8.1
zipp==3.11.0
zope.interface==5.5.2
  1. Create execution-environment.yml and fill it like this:
---
version: 3

build_arg_defaults:
  ANSIBLE_GALAXY_CLI_COLLECTION_OPTS: '--pre'

dependencies:
  ansible_core:
    package_pip: ansible-core==2.15.12
  ansible_runner:
    package_pip: ansible-runner==2.4.0
  galaxy: requirements.yml
  python: requirements.txt
  system: bindep.txt
  python_interpreter:
    package_system: "python3"
    python_path: "/usr/bin/python3.9"

images:
  base_image:
    name: docker.io/redhat/ubi9:latest

additional_build_files:
  - src: awx-24.6.1.tar.gz
    dest: dist

additional_build_steps:
  prepend_base:
     - RUN yum remove -y xmlsec1-1.2.29-9 xmlsec1-openssl-1.2.29-9
     - RUN yum install -y --allowerasing pkg-config https://repo.almalinux.org/almalinux/9/CRB/x86_64/os/Packages/libtool-ltdl-devel-2.4.6-45.el9.x86_64.rpm https://repo.almalinux.org/almalinux/8/AppStream/x86_64/os/Packages/xmlsec1-openssl-1.2.25-4.el8.x86_64.rpm https://repo.almalinux.org/almalinux/8/AppStream/x86_64/os/Packages/xmlsec1-1.2.25-4.el8.x86_64.rpm https://vault.centos.org/centos/8/PowerTools/x86_64/os/Packages/xmlsec1-devel-1.2.25-4.el8.x86_64.rpm
     - COPY _build/dist/awx-24.6.1.tar.gz /tmp/src/dist/awx.tar.gz
  prepend_builder:
    - ENV PKG_CONFIG_PATH=/usr/local/lib/x86_64-linux-gnu/pkgconfig:/usr/local/lib/pkgconfig:/usr/local/share/pkgconfig:/usr/lib/x86_64-linux-gnu/pkgconfig:/usr/lib/pkgconfig:/usr/share/pkgconfig
    - RUN /usr/bin/python3.9 -m pip install --upgrade pip
    - RUN yum remove -y python3-requests
  1. Run PATH=$PATH ansible-builder build -t awx-ee-test:latest --container-runtime=docker -vvv

Expected results

Successful build of the AWX component

Actual results

LookupError: setuptools-scm was unable to detect version for /tmp/pip-install-rdmhhdoj/awx_319cdf6fd3904dff9477393e4c1875ab

Additional information

No response

altbrace avatar Jul 30 '24 01:07 altbrace

https://github.com/ansible/awx/issues/11586 solved here

altbrace avatar Jul 30 '24 02:07 altbrace

Apparently not fixed

altbrace avatar Jul 30 '24 02:07 altbrace

So far it seems like people are running into issues when VERSION files are absent in the root and /awxkit directory

altbrace avatar Jul 30 '24 02:07 altbrace

I am slightly confused here. Apologies for the delay @altbrace but are you trying to include the awx collection in the EE? Or are you just building an EE to use inside of awx?

thedoubl3j avatar Jun 12 '25 15:06 thedoubl3j