Running Xcode Build

[maxlapides]

OS related issued, please help us identify the issue by posting the output of this

Linux max-ubuntu 5.8.0-63-generic #71~20.04.1-Ubuntu SMP Thu Jul 15 17:46:08 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux
1
NAME="Ubuntu"
PRETTY_NAME="Ubuntu 20.04.2 LTS"
VERSION_CODENAME=focal
UBUNTU_CODENAME=focal
Filesystem      Size  Used Avail Use% Mounted on
/dev/nvme0n1p1  938G  499G  392G  56% /
QEMU emulator version 4.2.1 (Debian 1:4.2-3ubuntu6.17)
Copyright (c) 2003-2019 Fabrice Bellard and the QEMU Project developers
libvirtd (libvirt) 6.0.0
              total        used        free      shared  buff/cache   available
Mem:           15Gi       2.6Gi       159Mi        75Mi        12Gi        12Gi
Swap:         2.0Gi        87Mi       1.9Gi
6
12
crw-rw----+ 1 root kvm 10, 232 Jul 23 14:29 /dev/kvm
total 8.0K
drwxrwxrwt  2 root       root       4.0K Jul 22 14:43 .
drwxrwxrwt 22 root       root       4.0K Jul 23 14:34 ..
srwxrwxrwx  1 maxlapides maxlapides    0 Jul 22 14:43 X0
srwxrwxr-x  1 gdm        gdm           0 Jul 22 14:43 X1024
srwxrwxr-x  1 gdm        gdm           0 Jul 22 14:43 X1025
root        1534  6.4  0.5 1538376 85236 ?       Ssl  Jul22  92:51 /usr/bin/dockerd -H fd:// --containerd=/run/containerd/containerd.sock
maxlapi+   56608  0.0  0.0  17672   740 pts/1    S+   14:35   0:00 grep --color=auto dockerd
kvm:x:108:
docker:x:998:maxlapides

Hey there! This is an awesome project. I've successfully created a deduped.img file following the CI/CD instructions.

I am now trying to use this .img file to run my Xcode build. So, here's what I'm running:

docker run -it \
    --device /dev/kvm \
    -p 50922:10022 \
    -e GENERATE_UNIQUE=true \
    -e NOPICKER=true \
    -e MASTER_PLIST_URL=https://raw.githubusercontent.com/sickcodes/Docker-OSX/master/custom/config-nopicker-custom.plist \
    -e IMAGE_PATH=/home/arch/OSX-KVM/mac_hdd_ng.img \
    -e "OSX_COMMANDS=/bin/bash -c \"pwd && uname -a\"" \
    mycontainer

This results in a (qemu) command prompt, but I'm not sure what to do with that. I thought the OSX_COMMANDS would be run and output, but that doesn't seem to happen here. Do you have any recommendations on what my next steps should be?

Thanks again! Really appreciate the work on this!

[maxlapides]

Not sure if this is helpful, but this is what I'm seeing!

[sickcodes]

-e "OSX_COMMANDS=/bin/bash -c \"pwd && uname -a\"" \

Is a demo command, you can change it to something else:

-e "OSX_COMMANDS=/bin/bash -c \"echo hello\""

The container will delete itself after running the commands, or you can remove that line

[maxlapides]

The container will delete itself after running the commands, or you can remove that line

I think this is where I'm a bit confused. Here's what I just ran:

docker run -it \
    --device /dev/kvm \
    -p 50922:10022 \
    -e GENERATE_UNIQUE=true \
    -e NOPICKER=true \
    -e MASTER_PLIST_URL=https://raw.githubusercontent.com/sickcodes/Docker-OSX/master/custom/config-nopicker-custom.plist \
    -e IMAGE_PATH=/home/arch/OSX-KVM/mac_hdd_ng.img \
    -e "OSX_COMMANDS=/bin/bash -c \"echo hello\"" \
    mycontainer

And here's the result:

It never terminates, and I never see "hello" logged. I feel like I'm so so close to having this working!

[sickcodes]

My container will only work in full auto with the OSX_COMMANDS if you make the user account arch and the password alpine.

Try that and you can use the full auto version, but you'll also need to build using the Dockerfile.auto

[maxlapides]

Okay I read through the Dockerfile.auto and I think I understand better now. In the above example, OSX_COMMANDS isn't working because I'm building my Docker image from :naked. Thanks for that tip!

FROM sickcodes/docker-osx:naked
USER arch
COPY --chown=arch ./deduped /image/mac_hdd_ng.img

So, now I'm trying a different approach where I run my docker-osx container by itself and then have my CI tasks SSH into that container to run various scripts. Locally, I am testing out SSH first by running this:

docker run -it \
  --device /dev/kvm \
  -p 50922:10022 \
  -e GENERATE_UNIQUE=true \
  -e NOPICKER=true \
  -e IMAGE_PATH=/image/mac_hdd_ng.img \
  mydockerosximage

And then after that starts up, outside of the container I can run:

sshpass -pmypassword ssh -o StrictHostKeyChecking=no [email protected] -p 50922 'uname -a'

And that works!

I can also run:

docker exec [MY_CONTAINER_ID] sshpass -pmypassword ssh -o StrictHostKeyChecking=no [email protected] -p 10022 'uname -a'

This also works 😄

So next I configured my Kubernetes-based CI system to use a readinessProbe to determine whether macOS is ready:

readinessProbe:
  exec:
    command: ["sshpass", "-pmypassword", "ssh", "-o", "StrictHostKeyChecking=no", "[email protected]", "-p", "10022", "'uname -a'"]

I also configured all the same environment variables:

env:
  - name: NOPICKER
    value: 'true'
  - name: GENERATE_UNIQUE
    value: 'true'
  - name: IMAGE_PATH
    value: '/image/mac_hdd_ng.img'

As expected, the probe fails until macOS is ready. It sends this error:

Readiness probe failed: ssh: connect to host 127.0.0.1 port 10022: Connection refused

But ultimately, the probe starts returning a different error:

Readiness probe failed: kex_exchange_identification: read: Connection reset by peer Connection reset by 127.0.0.1 port 10022

And that's where I'm currently stuck! This may be a configuration mistake in my CI system, but I'm not sure right now. Any ideas are welcome :)

[maxlapides]

Oh one more potentially useful clue, the logs for the docker-osx container end like this:

alsa: Could not initialize ADC
alsa: Failed to open `default':
alsa: Reason: No such file or directory
ALSA lib confmisc.c:855:(parse_card) cannot find card '0'
ALSA lib conf.c:5111:(_snd_config_evaluate) function snd_func_card_inum returned error: No such file or directory
ALSA lib confmisc.c:422:(snd_func_concat) error evaluating strings
ALSA lib conf.c:5111:(_snd_config_evaluate) function snd_func_concat returned error: No such file or directory
ALSA lib confmisc.c:1334:(snd_func_refer) error evaluating name
ALSA lib conf.c:5111:(_snd_config_evaluate) function snd_func_refer returned error: No such file or directory
ALSA lib conf.c:5599:(snd_config_expand) Evaluate error: No such file or directory
ALSA lib pcm.c:2660:(snd_pcm_open_noupdate) Unknown PCM default
alsa: Could not initialize ADC
alsa: Failed to open `default':
alsa: Reason: No such file or directory
audio: Failed to create voice `adc'
qemu-system-x86_64: Slirp: Failed to send packet, ret: -1
qemu-system-x86_64: Slirp: Failed to send packet, ret: -1
qemu-system-x86_64: Slirp: Failed to send packet, ret: -1
qemu-system-x86_64: Slirp: Failed to send packet, ret: -1
qemu-system-x86_64: Slirp: Failed to send packet, ret: -1
qemu-system-x86_64: Slirp: Failed to send packet, ret: -1
qemu-system-x86_64: Slirp: Failed to send packet, ret: -1
qemu-system-x86_64: Slirp: Failed to send packet, ret: -1
qemu-system-x86_64: Slirp: Failed to send packet, ret: -1
qemu-system-x86_64: Slirp: Failed to send packet, ret: -1
qemu-system-x86_64: Slirp: Failed to send packet, ret: -1

[sickcodes]

Remove OSX_COMMANDS to get a shell, only in :auto

Example: https://github.com/sickcodes/Docker-OSX#prebuilt-image-with-arbitrary-command-line-arguments

[Eslam-mohammed-anwar]

@maxlapides Did you end up running it for CICD pipelines , if you please share any heads up

[maxlapides]

@Eslam-mohammed-anwar I did eventually figure out the issue here. The root cause is that the readiness probe that I wrote is no good.