Privileged containers spawn ttys which conflict with host OS

[srstsavage]

TL;DR I think the footloose images need systemctl mask getty.target, or equivalent systemd files deleted.

When running some wksctl examples with Docker and Ubuntu 18.04, I experienced some odd behavior where the created containers would fight with the host OS (Debian 10) for control of ttys 1-6.

cat ubuntu1804/docker/singlemaster.yaml
cluster:
  name: ubuntu-singlemaster
  privateKey: cluster-key
machines:
- count: 2
  spec:
    image: quay.io/footloose/ubuntu18.04:0.6.3
    name: node%d
    portMappings:
    - containerPort: 22
      hostPort: 2222
    - containerPort: 6443
      hostPort: 6443
    # The below is required for dockerd to run smoothly.
    # See also: https://github.com/weaveworks/footloose#running-dockerd-in-container-machines
    privileged: true
    volumes:
    - type: volume
      destination: /var/lib/docker

$ footloose create -c ubuntu1804/docker/singlemaster.yaml
INFO[0000] Docker Image: quay.io/footloose/ubuntu18.04:0.6.3 present locally 
INFO[0000] Creating machine: ubuntu-singlemaster-node0 ... 
INFO[0000] Creating machine: ubuntu-singlemaster-node1 ... 
$ docker ps
CONTAINER ID        IMAGE                                 COMMAND             CREATED             STATUS              PORTS                                          NAMES
4a37e171739b        quay.io/footloose/ubuntu18.04:0.6.3   "/sbin/init"        3 seconds ago       Up 1 second         0.0.0.0:2223->22/tcp, 0.0.0.0:6444->6443/tcp   ubuntu-singlemaster-node1
0d8bd0e58f82        quay.io/footloose/ubuntu18.04:0.6.3   "/sbin/init"        3 seconds ago       Up 2 seconds        0.0.0.0:6443->6443/tcp, 0.0.0.0:2222->22/tcp   ubuntu-singlemaster-node0

After the footloose managed containers start up, every tty (tty1-tty6) has the login prompt. Some say my real hostname, some say node0, some say node1. Also only every third keypress on my physical keyboard seems to be received because the node0 and node1 containers are consuming the other two. If i shut down one of the footloose managed containers, every other keypress is consumed.

The cause of the issue is apparently systemd inside the containers spinning up getty services. This only happens when containers are run in priviledged mode (Ubuntu/Debian) (note [email protected] below).

docker exec -it ubuntu-singlemaster-node0 systemctl status 
● node0
    State: running
     Jobs: 0 queued
   Failed: 0 units
    Since: Thu 2020-04-02 19:50:11 UTC; 1s ago
   CGroup: /system.slice/containerd.service
           ├─init.scope
           └─system.slice
             ├─systemd-journald.service
             ├─ssh.service
             ├─systemd-resolved.service
             ├─dbus.service
             ├─system-getty.slice
             │ └─[email protected]
             │   └─30 /sbin/agetty -o -p -- \u --noclear tty1 linux
             └─systemd-logind.service

When containers first run systemd is usually in a degraded state because the getty service failed to start.

$ docker exec -it ubuntu-singlemaster-node1 systemctl status 
● node1
    State: degraded
     Jobs: 0 queued
   Failed: 1 units
    Since: Thu 2020-04-02 20:09:12 UTC; 2min 29s ago
   CGroup: /system.slice/containerd.service
           ├─init.scope
           └─system.slice
             ├─systemd-journald.service
             ├─ssh.service
             ├─systemd-resolved.service
             ├─dbus.service
             └─systemd-logind.service

The fix appears to be masking getty.target. Example Dockerfile inheriting footloose image:

FROM quay.io/footloose/ubuntu18.04:0.6.3

#disable getty so it doesn't conflict with the host OS
RUN systemctl mask getty.target

A better solution would be to include getty.target in the list of targets masked in the footloose images, e.g. here in the Ubuntu 18.04 image.

Does this make sense? Any reason these images need systemd managed getty?

[rwanyoike]

thanks - 😅 had this happen on the host (archlinux). thought I broke something. only way I could recover was to ssh in and stop docker