eks-anywhere-build-tooling icon indicating copy to clipboard operation
eks-anywhere-build-tooling copied to clipboard

Qemu instance fails to reboot during build operations

Open eccentrichead opened this issue 1 year ago • 0 comments

What happened: When using image-builder to build an ubuntu22.04 image with kubernetes 1.28 release, the build process remains stuck post the Qemu instance boot up. If left unattended at this stage, the process eventually times outs waiting for the SSH connection to appear and subsequent ansible playbook execution fails causing the whole operation to fail.

What you expected to happen: The Qemu build instance should reboot automatically during the build operation once the packages are installed.

How to reproduce it (as minimally and precisely as possible):

  • On a baremetal hardware setup the environment with image-builder utility: https://anywhere.eks.amazonaws.com/docs/osmgmt/artifacts/#build-bare-metal-node-images
  • Setup the required customization by providing the file.json if needed.
  • install the ssviewer tool to connect to VNC terminal when the qemu instance is spawned
  • Run the build tool with command:
$ image-builder build --os ubuntu --os-version 22.04 --hypervisor baremetal --release-channel 1-28 --files-config file.json -v8
  • Login into the Qemu instance with ssviewer to the 127.0.0x:xxxx ip port.
  • The process in side qemu instance will get stuck post the CHANGE ens4 messages and the whole build operation will continue to wait. qemu-terminal
  • At this point if we wait, the process inside the qemu instance will timeout and drop to shell. In the shell a simple reboot command will cause the next step of build process to execute and complete successfully with the required image generated. Anything else we need to know?:

Environment:

  • EKS Anywhere Release: 0.18.2
  • EKS Distro Release: ubuntu 22.04

eccentrichead avatar Aug 07 '24 05:08 eccentrichead