Can't integrate flannel and docker when flannel runs in kubernetes

[Alexhha]

I have read https://github.com/coreos/flannel/blob/master/Documentation/running.md#docker-integration and have tried to integrate docker (17.03/Ubuntu 16.04) with flannel (latest) but without success

Expected Behavior

Docker should use ${FLANNEL_SUBNET/${FLANNEL_MTU} environment variables

Current Behavior

After reboot docker doesn't start

Jan 21 18:03:52 k8s-node2 systemd[1]: docker.service: Failed to load environment files: No such file or director
Jan 21 18:03:52 k8s-node2 systemd[1]: docker.service: Failed to run 'start' task: No such file or directory
Jan 21 18:03:52 k8s-node2 systemd[1]: Failed to start Docker Application Container Engine.
Jan 21 18:03:52 k8s-node2 systemd[1]: docker.service: Failed with result 'resources'.

Possible Solution

I'm not sure but it seems that flannel must be installed to docker host itself? We can't use dockerized version?

Steps to Reproduce (for bugs)

1. Configure kubernetes cluster (1.8.x) with kubeadm

# kubeadm init --pod-network-cidr=10.244.0.0/16 --apiserver-advertise-address=192.168.1.103  --kubernetes-version stable-1.8 --node-name k8s-master --token-ttl 0

2. Apply flannel

$ kubectl apply -f https://raw.githubusercontent.com/coreos/flannel/master/Documentation/kube-flannel.yml

3. Copy docker system unit (/lib/systemd/system/docker.service) to the /etc/systemd/system/docker.service amd modify it

# cat /etc/systemd/system/docker.service
[Unit]
Description=Docker Application Container Engine
Documentation=https://docs.docker.com
After=network.target docker.socket firewalld.service
Requires=docker.socket

[Service]
Type=notify

EnvironmentFile=/var/run/flannel/subnet.env
ExecStart=/usr/bin/dockerd \
    --host fd:// \
    --bip=${FLANNEL_SUBNET} \
    --mtu=${FLANNEL_MTU}

ExecReload=/bin/kill -s HUP $MAINPID

LimitNOFILE=1048576
LimitNPROC=infinity
LimitCORE=infinity
TasksMax=infinity
TimeoutStartSec=0
Delegate=yes
KillMode=process

RuntimeDirectory=docker

[Install]
WantedBy=multi-user.target

4. Restart docker

# systemctl daemon-reload
# systemctl restart docker

5. Check

Context

After system restart - docker doesn't start

Jan 21 18:03:52 k8s-node2 systemd[1]: Stopped Docker Application Container Engine.
Jan 21 18:03:52 k8s-node2 systemd[1]: docker.service: Failed to load environment files: No such file or director
Jan 21 18:03:52 k8s-node2 systemd[1]: docker.service: Failed to run 'start' task: No such file or directory
Jan 21 18:03:52 k8s-node2 systemd[1]: Failed to start Docker Application Container Engine.
Jan 21 18:03:52 k8s-node2 systemd[1]: docker.service: Failed with result 'resources'.

Your Environment

• Flannel version: latest
• Backend used (e.g. vxlan or udp):
• Etcd version:
• Kubernetes version (if used): 1.8.7
• Operating System and version: Ubuntu 16.04.3 LTS

[Alexhha]

Any advice?

[jscottsf]

Would also like some advice on this.

There seems to be a lot of conflicting info on the web on how to exactly setup containerized Flannel via kubeadm. Mainly, do you need to further edit the docker.service to use any generated env vars (which would not be available until you first run Docker to get Flannel running in a container). Chicken and egg problem?

Lastly, I'm not getting DNS resolution working from within containers. Completely tangental to this issue.

[fntlnz]

Related to: https://github.com/kubernetes/kubernetes/issues/70202

[joestump]

I recently ran into this issue as well running Flannel as a DaemonSet. After flanneld is running the /run/flannel/subnet.env file exists. I'd originally set up my Docker unit file to just use EnvironmentFile=/run/flannel/subnet.env, but ran into the same issue @Alexhha ran into when I rebooted the nodes.

After checking that subnet.env was the same after restarts I copied it into /etc/docker/flannel.env and restarted my worker nodes. Unfortunately, this led to this issue where basically the Docker bridge network was incorrect.

What I'm stuck with now is running ifconfig docker0 down && brctl delbr docker0 && systemctl restart docker after a node is restarted. I think this is basically a duplicate of kubernetes/kubernetes#70202, but the reason it was closed isn't correct. The subnet.env file does show up, it just shows up after flanneld is running. As @jansmets points out in kubernetes/kubernetes#70202, it's a 🐔 and 🥚 between the Systemd controlled dockerd and the DaemonSet for Flannel.

I do see a flanneld running on each node though, which makes me wonder if I can use a Systemd approach similar to this to have dockerd wait on /run/flannel/subnet.env to exist before starting. Though I'm guessing dockerd is needed to even start the DaemonSet.

[stale[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.