cppagent
cppagent copied to clipboard
mtconnect
Alpine docker build fails
Formal issue stemming from discussion here: https://github.com/orgs/mtconnect/discussions/476
Only seems to affect Alpine/arm64.
Alpine w/o buildx works fine. Ubuntu images build successfully.
Alpine docker build fails. Here is the github action error from my own fork:
#15 146.5 boost/1.85.0: RUN: b2 -q target-os=linux architecture=x86 address-model=64 binary-format=elf abi=sysv --layout=system --user-config=/root/.conan2/p/boost5fdc72773338c/s/src/tools/build/user-config.jam -sNO_ZLIB=0 -sNO_BZIP2=0 -sNO_LZMA=1 -sNO_ZSTD=1 boost.locale.icu=off --disable-icu boost.locale.iconv=on boost.locale.iconv.lib=libc threading=multi visibility=hidden link=static variant=release --with-atomic --with-charconv --with-chrono --with-container --with-context --with-coroutine --with-date_time --with-exception --with-filesystem --with-iostreams --with-json --with-locale --with-log --with-program_options --with-random --with-regex --with-stacktrace --with-system --with-thread --with-timer --with-type_erasure --with-url toolset=gcc cxxstd=17 cxxstd-dialect=gnu define=_GLIBCXX_USE_CXX11_ABI=1 pch=on -sLIBBACKTRACE_PATH=/root/.conan2/p/b/libbab04427202a521/p linkflags="" cxxflags="-fPIC -DBOOST_STACKTRACE_ADDR2LINE_LOCATION=/usr/bin/addr2line" install --prefix=/root/.conan2/p/b/boostdd77d3b1dab0b/p -j2 --abbreviate-paths -d0 --debug-configuration --build-dir="/root/.conan2/p/b/boostdd77d3b1dab0b/b/build-release"
#15 146.5 Assertion failed: destlen >= total (libgcompat/string.c: __strncat_chk: 251)
#15 146.6
#15 146.6 boost/1.85.0: ERROR:
#15 146.6 Package '74482cd480d12051a8a314bff4f706866d4488a6' build failed
#15 146.6 boost/1.85.0: WARN: Build folder /root/.conan2/p/b/boostdd77d3b1dab0b/b/build-release
#15 146.6 ERROR: boost/1.85.0: Error in build() method, line 1130
#15 146.6 self.run(full_command)
#15 146.6 ConanException: Error -6 while executing
#15 ERROR: process "/bin/sh -c if [ -z \"$WITH_TESTS\" ] || [ \"$WITH_TESTS\" = \"false\" ]; then WITH_TESTS_ARG=\"--test-folder=\"; else WITH_TESTS_ARG=\"\"; fi && . /python/conan/bin/activate && conan profile detect && conan create cppagent --build=missing -c \"tools.build:jobs=$CONAN_CPU_COUNT\" -o agent_prefix=mtc -o cpack=True -o \"with_ruby=$WITH_RUBY\" -o cpack_destination=/root/agent -o cpack_name=dist -o cpack_generator=TGZ -pr \"$CONAN_PROFILE\" \"${WITH_TESTS_ARG}\" && tar xf dist.tar.gz && rm dist.tar.gz" did not complete successfully: exit code: 1
#18 [linux/arm64 build 4/4] RUN if [ -z "false" ] || [ "false" = "false" ]; then WITH_TESTS_ARG="--test-folder="; else WITH_TESTS_ARG=""; fi && . /python/conan/bin/activate && conan profile detect && conan create cppagent --build=missing -c "tools.build:jobs=2" -o agent_prefix=mtc -o cpack=True -o "with_ruby=True" -o cpack_destination=/root/agent -o cpack_name=dist -o cpack_generator=TGZ -pr "/root/agent/cppagent/conan/profiles/docker" "argument" && tar xf dist.tar.gz && rm dist.tar.gz
#18 CANCELED
------
> [linux/amd64 build 4/4] RUN if [ -z "false" ] || [ "false" = "false" ]; then WITH_TESTS_ARG="--test-folder="; else WITH_TESTS_ARG=""; fi && . /python/conan/bin/activate && conan profile detect && conan create cppagent --build=missing -c "tools.build:jobs=2" -o agent_prefix=mtc -o cpack=True -o "with_ruby=True" -o cpack_destination=/root/agent -o cpack_name=dist -o cpack_generator=TGZ -pr "/root/agent/cppagent/conan/profiles/docker" "argument" && tar xf dist.tar.gz && rm dist.tar.gz:
146.5 boost/1.85.0: WARN: b2 -q target-os=linux architecture=x86 address-model=64 binary-format=elf abi=sysv --layout=system --user-config=/root/.conan2/p/boost5fdc72773338c/s/src/tools/build/user-config.jam -sNO_ZLIB=0 -sNO_BZIP2=0 -sNO_LZMA=1 -sNO_ZSTD=1 boost.locale.icu=off --disable-icu boost.locale.iconv=on boost.locale.iconv.lib=libc threading=multi visibility=hidden link=static variant=release --with-atomic --with-charconv --with-chrono --with-container --with-context --with-coroutine --with-date_time --with-exception --with-filesystem --with-iostreams --with-json --with-locale --with-log --with-program_options --with-random --with-regex --with-stacktrace --with-system --with-thread --with-timer --with-type_erasure --with-url toolset=gcc cxxstd=17 cxxstd-dialect=gnu define=_GLIBCXX_USE_CXX11_ABI=1 pch=on -sLIBBACKTRACE_PATH=/root/.conan2/p/b/libbab04427202a521/p linkflags="" cxxflags="-fPIC -DBOOST_STACKTRACE_ADDR2LINE_LOCATION=/usr/bin/addr2line" install --prefix=/root/.conan2/p/b/boostdd77d3b1dab0b/p -j2 --abbreviate-paths -d0 --debug-configuration --build-dir="/root/.conan2/p/b/boostdd77d3b1dab0b/b/build-release"
146.5 boost/1.85.0: RUN: b2 -q target-os=linux architecture=x86 address-model=64 binary-format=elf abi=sysv --layout=system --user-config=/root/.conan2/p/boost5fdc72773338c/s/src/tools/build/user-config.jam -sNO_ZLIB=0 -sNO_BZIP2=0 -sNO_LZMA=1 -sNO_ZSTD=1 boost.locale.icu=off --disable-icu boost.locale.iconv=on boost.locale.iconv.lib=libc threading=multi visibility=hidden link=static variant=release --with-atomic --with-charconv --with-chrono --with-container --with-context --with-coroutine --with-date_time --with-exception --with-filesystem --with-iostreams --with-json --with-locale --with-log --with-program_options --with-random --with-regex --with-stacktrace --with-system --with-thread --with-timer --with-type_erasure --with-url toolset=gcc cxxstd=17 cxxstd-dialect=gnu define=_GLIBCXX_USE_CXX11_ABI=1 pch=on -sLIBBACKTRACE_PATH=/root/.conan2/p/b/libbab04427202a521/p linkflags="" cxxflags="-fPIC -DBOOST_STACKTRACE_ADDR2LINE_LOCATION=/usr/bin/addr2line" install --prefix=/root/.conan2/p/b/boostdd77d3b1dab0b/p -j2 --abbreviate-paths -d0 --debug-configuration --build-dir="/root/.conan2/p/b/boostdd77d3b1dab0b/b/build-release"
146.5 Assertion failed: destlen >= total (libgcompat/string.c: __strncat_chk: 251)
146.6
146.6 boost/1.85.0: ERROR:
146.6 Package '74482cd480d12051a8a314bff4f706866d4488a6' build failed
146.6 boost/1.85.0: WARN: Build folder /root/.conan2/p/b/boostdd77d3b1dab0b/b/build-release
146.6 ERROR: boost/1.85.0: Error in build() method, line 1130
146.6 self.run(full_command)
146.6 ConanException: Error -6 while executing
------
1 warning found (use --debug to expand):
- UndefinedVar: Usage of undefined variable '$HOME' (line 47)
Dockerfile:80
--------------------
79 | # Build and optionally test. Unpack the dist to reduce overhead in the following release step.
80 | >>> RUN if [ -z "$WITH_TESTS" ] || [ "$WITH_TESTS" = "false" ]; then \
81 | >>> WITH_TESTS_ARG="--test-folder="; \
82 | >>> else \
83 | >>> WITH_TESTS_ARG=""; \
84 | >>> fi \
85 | >>> && . /python/conan/bin/activate \
86 | >>> && conan profile detect \
87 | >>> && conan create cppagent \
88 | >>> --build=missing \
89 | >>> -c "tools.build:jobs=$CONAN_CPU_COUNT" \
90 | >>> -o agent_prefix=mtc \
91 | >>> -o cpack=True \
92 | >>> -o "with_ruby=$WITH_RUBY" \
93 | >>> -o cpack_destination=/root/agent \
94 | >>> -o cpack_name=dist \
95 | >>> -o cpack_generator=TGZ \
96 | >>> -pr "$CONAN_PROFILE" \
97 | >>> "${WITH_TESTS_ARG}" \
98 | >>> && tar xf dist.tar.gz \
99 | >>> && rm dist.tar.gz
100 |
--------------------
ERROR: failed to solve: process "/bin/sh -c if [ -z \"$WITH_TESTS\" ] || [ \"$WITH_TESTS\" = \"false\" ]; then WITH_TESTS_ARG=\"--test-folder=\"; else WITH_TESTS_ARG=\"\"; fi && . /python/conan/bin/activate && conan profile detect && conan create cppagent --build=missing -c \"tools.build:jobs=$CONAN_CPU_COUNT\" -o agent_prefix=mtc -o cpack=True -o \"with_ruby=$WITH_RUBY\" -o cpack_destination=/root/agent -o cpack_name=dist -o cpack_generator=TGZ -pr \"$CONAN_PROFILE\" \"${WITH_TESTS_ARG}\" && tar xf dist.tar.gz && rm dist.tar.gz" did not complete successfully: exit code: 1
Error: buildx failed with: ERROR: failed to solve: process "/bin/sh -c if [ -z \"$WITH_TESTS\" ] || [ \"$WITH_TESTS\" = \"false\" ]; then WITH_TESTS_ARG=\"--test-folder=\"; else WITH_TESTS_ARG=\"\"; fi && . /python/conan/bin/activate && conan profile detect && conan create cppagent --build=missing -c \"tools.build:jobs=$CONAN_CPU_COUNT\" -o agent_prefix=mtc -o cpack=True -o \"with_ruby=$WITH_RUBY\" -o cpack_destination=/root/agent -o cpack_name=dist -o cpack_generator=TGZ -pr \"$CONAN_PROFILE\" \"${WITH_TESTS_ARG}\" && tar xf dist.tar.gz && rm dist.tar.gz" did not complete successfully: exit code: 1
Github Copliot tells me to check if this Boost version is compatible with this Conan version? 🤷
Any other ideas?
Builds fine using build and on Ubuntu, so I dont think it has anything to do with the boost version. The boost version is also coming from ConanCentral so it has been tested on all the platforms. It also passes the CI/CD tests on three platforms.
I think this is a buildx bug. It sometimes dies in the libxml or other builds. Rather random.
Using build for Alpine docker image works fine on the host arch. I think this is a qemu issue with alpine when using buildx. That is as much as I can surmise at this point.
Try changing qemu versions or docket buildx. That's where I left off. Went through all the obvious settings like you, no luck. I think it is the cross compile environment that is killing the build. Can't think of anything else.
i must be reading my action's logs wrong; im reading that my runs are failing building linux/amd64?
yup - its the amd64 build that is failing, not arm64. Which is even weirder. cross-compile works, but native arch doesnt? 🤔 🤷
I removed linux/amd64 in my workflow file and it built successfully:
# build docker image and push to docker hub
# note: this won't work because the cross-platform build takes around 10h,
# but github limits us to 6h.
# so need to build it locally and push to docker hub.
# see Dockerfile for instructions.
# needs github secrets set for -
# DOCKERHUB_USERNAME - ie mtconnect
# DOCKERHUB_TOKEN - token for that account
# Set VERSION below before building.
# See CMakeLists.txt for the current version.
# Note: this will tag the image with that version, as well as 'latest'.
name: Build Docker image
env:
PLATFORMS: linux/arm64
# when to run workflow
on:
# # run on push or pull request events for the master branch
push:
tags:
- "v*.*.*"
# allows you to run this workflow manually from the actions tab
workflow_dispatch:
# a workflow run has one or more jobs that can run sequentially or in parallel
jobs:
# this workflow contains a single job called "build"
build:
# the type of runner that the job will run on
runs-on: ubuntu-latest
strategy:
max-parallel: 1
matrix:
include:
- dockerfile: docker/alpine/Dockerfile
name: agent
# - dockerfile: demo/Dockerfile
# name: demo
# steps represent a sequence of tasks that will be executed as part of the job
steps:
# checks-out your repository under $GITHUB_WORKSPACE
# see https://github.com/actions/checkout
- name: Checkout Agent
uses: actions/checkout@v3
# the QEMU emulator lets us build for arm processors also
# see https://github.com/docker/setup-qemu-action
- name: Set up QEMU emulator
uses: docker/setup-qemu-action@v3
# see https://github.com/docker/setup-buildx-action
- name: Set up Docker Buildx
uses: docker/setup-buildx-action@v3
- name: Login to DockerHub
uses: docker/login-action@v3
with:
username: ${{ secrets.DOCKERHUB_USERNAME }}
password: ${{ secrets.DOCKERHUB_TOKEN }}
- name: Remove leading v from the version
run: |
echo "VERSION=$(echo ${{ github.ref_name }} | sed 's/^v//')" >> $GITHUB_ENV
echo "MAJOR_VERSION=$(echo ${{ github.ref_name }} | sed -E 's/^v([0-9]+)\.([0-9]+)\..+/\1.\2/')" >> $GITHUB_ENV
# see https://github.com/docker/build-push-action
- name: Build image and push to DockerHub
uses: docker/build-push-action@v5
with:
context: .
file: ${{ matrix.dockerfile }}
platforms: ${{ env.PLATFORMS }}
# docker hub user/repo:tag
tags: |
${{ secrets.DOCKERHUB_USERNAME }}/${{ matrix.name }}:latest
${{ secrets.DOCKERHUB_USERNAME }}/${{ matrix.name }}:${{ env.VERSION }}
${{ secrets.DOCKERHUB_USERNAME }}/${{ matrix.name }}:${{ env.MAJOR_VERSION }}
# push to docker hub
push: true
Not sure if we can build on GitHub using ARM architecture and then push using build instead of buildx.
Not sure if we can build on GitHub using ARM architecture and then push using build instead of buildx.
We looked into that recently, I don't believe github offers anything other than amd64 runners. You can host your own runner, which can be arm. However that all but defeats the point of using github actions cloud
Looks like public github-hosted arm runners are coming "this year" :
https://github.com/orgs/community/discussions/73040
Looks like they entered private beta in Jan. Maybe the my are now available.
we are building both amd and arm, into alpine images, with qemu on github for our other projects, and they build fine. so this is something to do with conan/boost build process in qemu
For now, I am building the alpine image for arm64 in my fork since it appears the amd64 is what is failing, arm64 seems to build fine.
if anyone wants/needs alpine image sizes for arm64, you can find it here: https://hub.docker.com/repository/docker/robotranger/agent/general
Cross compiling in docker is broken for the moment in alpine. I have tried to work around it and have had no success. If you find a solution, please post it.
The crazy thing is the cross compiling is the part that is working!
ARM64 builds fine.
AMD64 is failing.
Just so I don't duplicate work, have we tried building AMD WITHOUT qemu?
found this the other day: https://learn.arm.com/learning-paths/cross-platform/github-arm-runners/actions/
i ran a build with both arch, buildx and qemu today. used build-docker-image.yml with the following changes:
- un-commented
,linux/arm64 - replaced
- dockerfile: docker/ubuntu/Dockerfilewith- dockerfile: docker/alpine/Dockerfile
seems to work! build succeeded, I see both arch in docker hub:
love to see that 30mb 😎👍
I just discovered the same thing. I’m going to change back to alpine and rerun the docker workflow. I need to tag first.BestW(Sent from mobile)On May 29, 2025, at 19:18, robot-ranger @.***> wrote:robot-ranger left a comment (mtconnect/cppagent#480) i ran a build with both arch, buildx and qemu today. used build-docker-image.yml with the following changes:
un-commented ,linux/arm64 replaced - dockerfile: docker/ubuntu/Dockerfile with - dockerfile: docker/alpine/Dockerfile
seems to work! build succeeded, I see both arch in docker hub: image.png (view on web) love to see that 30mb 😎👍
—Reply to this email directly, view it on GitHub, or unsubscribe.You are receiving this because you commented.Message ID: @.***>
Currently running a build with github-hosted arm runner. building "native" images; no QEMU involved (commented out the whole qemu step)
both arch are well into the build phase, seems to be working:
# a workflow run has one or more jobs that can run sequentially or in parallel
jobs:
# this workflow contains a single job called "build"
build:
# Do not build if it is an RC build
if: ${{ ! contains(github.ref_name, 'RC') }}
strategy:
max-parallel: 2
matrix:
platform: [ubuntu-22.04-arm, ubuntu-latest]
# the type of runner that the job will run on
runs-on: ${{ matrix.platform }}
Edit: it worked! no qemu!
13minute build. 😮 it was previously taking >3hrs!!!
@wsobel - any thoughts on this? i think we should leverage github's hosted arm-native runners now that theyre available. eliminates qemu and reduces build time by 92% (3hr > 13min)
It appears to be working now. I switched back to alpine and have been building on GitHub for the last couple of releases.
I'm not concerned about time since it only fires on a release and I don't care if it takes 2 hours.
strategy:
max-parallel: 1
matrix:
include:
- dockerfile: docker/alpine/Dockerfile
name: agent
- dockerfile: demo/Dockerfile
name: demo