在Centos7上部署kubernetes集群

Tags: 印象笔记

[toc]

0. 准备工作

安装工作需要从google下载docker镜像，所以需要代理或者能够正常访问google的网络环境，参考之前的《shadowsocks + privoxy代理配置》。
关闭SELinux，kubernetes暂时无法解决SELinux的问题。
关闭防火墙和iptables。

1. 配置yum源

docker的yum源：

$ cat <<EOF > /etc/yum.repos.d/docker.repo
[dockerrepo]
name=Docker Repository
baseurl=https://yum.dockerproject.org/repo/main/centos/7/
enabled=1
gpgcheck=1
gpgkey=https://yum.dockerproject.org/gpg
EOF

kubernetes的yum源：

$ cat <<EOF > /etc/yum.repos.d/kubernetes.repo
[kubernetes]
name=Kubernetes
baseurl=http://yum.kubernetes.io/repos/kubernetes-el7-x86_64
enabled=1
gpgcheck=1
repo_gpgcheck=1
gpgkey=https://packages.cloud.google.com/yum/doc/yum-key.gpg
       https://packages.cloud.google.com/yum/doc/rpm-package-key.gpg
EOF

2. 配置yum和docker服务使用代理

yum使用代理，修改/etc/yum.conf文件，增加如下内容：

proxy=http://<proxy ip>:<proxy port>

有些repo是不希望启用代理的，比如本地或者国内直连比较快的，在对应的repo配置下面增加一行：

proxy=_none_

docker使用代理：

mkdir /etc/systemd/system/docker.service.d
echo '[Service]' > /etc/systemd/system/docker.service.d/http-proxy.conf
echo 'Environment="HTTP_PROXY=http://<proxy ip>:<proxy port>/" "NO_PROXY=localhost,127.0.0.1,docker.jcing.com"' >> /etc/systemd/system/docker.service.d/http-proxy.conf
systemctl daemon-reload
# 检查输出
systemctl show --property=Environment docker
systemctl restart docker

3. yum安装docker与kubernetes

安装docker：

$ yum install -y docker-engine
$ systemctl enable docker && systemctl start docker

安装kubernetes：

# 关闭SELinux
$ setenforce 0
$ yum install -y kubelet kubeadm kubectl kubernetes-cni
$ systemctl enable kubelet && systemctl start kubelet

4. 集群初始化

确保shell中没有设置http_proxy和https_proxy环境变量，然后执行下面的初始化命令：

$ kubeadm init

kubernetes会自动从google服务器中下载相关的docker镜像，大致输出如下：

<master/tokens> generated token: "f0c861.753c505740ecde4c"
<master/pki> created keys and certificates in "/etc/kubernetes/pki"
<util/kubeconfig> created "/etc/kubernetes/kubelet.conf"
<util/kubeconfig> created "/etc/kubernetes/admin.conf"
<master/apiclient> created API client configuration
<master/apiclient> created API client, waiting for the control plane to become ready
<master/apiclient> all control plane components are healthy after 61.346626 seconds
<master/apiclient> waiting for at least one node to register and become ready
<master/apiclient> first node is ready after 4.506807 seconds
<master/discovery> created essential addon: kube-discovery
<master/addons> created essential addon: kube-proxy
<master/addons> created essential addon: kube-dns

Kubernetes master initialised successfully!

You can connect any number of nodes by running:

kubeadm join --token <token> <master-ip>

如果卡在<master/apiclient> created API client, waiting for the control plane to become ready这一步，请检查网络，很有可能是无法从服务器拉取镜像所导致的。一个简单的测试代理是否可用的方法：

# 下面这一步会卡死
$ curl www.google.com
# 下面的会有返回
$ http_proxy=http://<proxy ip>:<proxy port> curl www.google.com

如果卡在<master/apiclient> waiting for at least one node to register and become ready或者<master/discovery> created essential addon: kube-discovery，可以通过systemctl status kubelet查看信息，也许能发现卡住的原因。

输出的最后一行很重要，kubeadm join --token <token> <master-ip>，请记录。

如果集群的master上也可以部署pod，执行下面的命令：

$ kubectl taint nodes --all dedicated-
node "test-01" tainted
taint key="dedicated" and effect="" not found.
taint key="dedicated" and effect="" not found.

如果某部配置失败，需要清理环境重新执行：

# 查看已经存在的的namespace并删除
$ kubectl get namespace
$ kubectl delete namespace sock-shop
# 清理环境
$ kubeadm reset
# 重新执行
$ systemctl start kubelet.service
$ kubeadm init

5. 部署pod网络

$ kubectl apply -f https://git.io/weave-kube

6. 向集群中加入节点

节点也需要能够正常访问google服务的，按照0、1、2步做好准备工作后，执行命令：

# 这一行就是kubeadm init命令的最后一行输出，token是加入秘钥，master-ip是集群master的IP
$ kubeadm join --token <token> <master-ip>
<util/tokens> validating provided token
<node/discovery> created cluster info discovery client, requesting info from "http://138.68.156.129:9898/cluster-info/v1/?token-id=0f8588"
<node/discovery> cluster info object received, verifying signature using given token
<node/discovery> cluster info signature and contents are valid, will use API endpoints [https://138.68.156.129:443]
<node/csr> created API client to obtain unique certificate for this node, generating keys and certificate signing request
<node/csr> received signed certificate from the API server, generating kubelet configuration
<util/kubeconfig> created "/etc/kubernetes/kubelet.conf"

Node join complete:
* Certificate signing request sent to master and response
  received.
* Kubelet informed of new secure connection details.

Run 'kubectl get nodes' on the master to see this machine join.
# 检查集群中的节点
$ kubectl get nodes

7. 安装测试应用

装完没什么用，可以不装，跳过这一步。

# 新建namespace
$ kubectl create namespace sock-shop
# 安装应用
$ kubectl apply -n sock-shop -f "https://github.com/microservices-demo/microservices-demo/blob/master/deploy/kubernetes/complete-demo.yaml?raw=true"
# 查看应用
$ kubectl describe svc front-end -n sock-shop
Name:                   front-end
Namespace:              sock-shop
Labels:                 name=front-end
Selector:               name=front-end
Type:                   NodePort
IP:                     100.66.88.176
Port:                   <unset> 80/TCP
NodePort:               <unset> 31869/TCP
Endpoints:              <none>
Session Affinity:       None
$ kubectl get pods -n sock-shop
NAME                            READY     STATUS    RESTARTS   AGE
cart-2013512370-0ii5d           1/1       Running   2          4d
cart-db-1445314776-1opbe        1/1       Running   0          1h
catalogue-3777349842-ko0ps      1/1       Running   2          4d
catalogue-db-2196966982-jihxd   1/1       Running   0          1h
front-end-697319832-7mlos       1/1       Running   2          4d
orders-3580282209-urdkx         1/1       Running   0          1h
orders-db-1215677090-9g5ui      1/1       Running   2          4d
payment-1376044718-1lej0        1/1       Running   0          1h
queue-master-1190579278-yvm1a   1/1       Running   2          4d
rabbitmq-1897447621-wymmw       1/1       Running   0          1h
shipping-589875162-30gg9        1/1       Running   2          4d
user-3338781425-bsyc0           1/1       Running   2          4d
user-db-710789251-32e0j         1/1       Running   0          1h

8. Dashboard

8.1 安装Dashboard

Dashboard是kubernetes的网页端可视化界面，配置的前提是kube-dns和weave网络启动正常。

# 安装dashboard
$ kubectl create -f https://rawgit.com/kubernetes/dashboard/master/src/deploy/kubernetes-dashboard.yaml

因为使用kubeadm安装的集群是不带认证的，所以无法直接从https://<master ip>/ui访问。但是可以找到暴露的本机IP：

$ kubectl describe services kubernetes-dashboard -n kube-system
Name:                   kubernetes-dashboard
Namespace:              kube-system
Labels:                 app=kubernetes-dashboard
Selector:               app=kubernetes-dashboard
Type:                   NodePort
IP:                     10.107.143.60
Port:                   <unset> 80/TCP
NodePort:               <unset> 30353/TCP
Endpoints:              10.32.0.24:9090
Session Affinity:       None
No events.#

找到NodePort，然后直接访问http://<master ip>:30353就能够进入dashboard。参考这个Issue。

8.2 监控cpu和memory

Dashboard的github README上介绍说，要启用图形化监控cpu和内存，需要在集群上运行heapster。

git clone https://github.com/kubernetes/heapster.git ~/heapster
cd ~/heapster
kubectl apply -f deploy/kube-config/influxdb/

文档上说也要安装deploy/kube-config/google下的东西，我们是不需要的，只要安装了influxdb就可以了。

ps: 如果需要卸载，运行kubectl delete -f deploy/kube-config/influxdb就行。其他按照文件安装的pod也可以使用“delete -f”参数来删除。

参考资料

使用kubeadm安装 #kubernetes #官方文档 Dashboard #kubernetes #官方文档 #dashboard 在centos上安装docker #docker #官方文档为docker配置http代理 #docker #官方文档 Dashboard #kubernetes #github #dashboard Dashboard关于“Unauthorized”的Issue #kubernetes #github #dashboard Heapster # kubernetes #github #dashboard

Dec 22 '16 01:12 qianlei90

这个教程完全部署不了

Jul 30 '18 10:07 charmby

@charmby 哪里有问题？官方的kubeadm命令基本上没变过，应该还都是适用的

Jul 30 '18 10:07 qianlei90

kubernetes-dashboard 的設定改了 https://github.com/kubernetes/dashboard/wiki/Installation

Aug 10 '18 07:08 cycgp

Blog Blog copied to clipboard