Kubeadm部署Kubernetes1.14.3集群

一、環(huán)境說明

  主機名       IP地址            角色          系統(tǒng)       
  node11    192.168.11.11   k8s-master  Centos7.6
  node12    192.168.11.12   k8s-node    Centos7.6
  node13    192.168.11.13   k8s-node    Centos7.6

注意：官方建議每臺機器至少雙核2G內(nèi)存

以下命令在三臺主機上均需運行

1、設(shè)置阿里云yum源（可選）

curl -o /etc/yum.repos.d/CentOS-Base.repo http://mirrors.aliyun.com/repo/Centos-7.repo

2、安裝依賴包

yum install -y epel-release conntrack ipvsadm ipset jq sysstat curl iptables libseccomp

3、關(guān)閉防火墻

systemctl stop firewalld && systemctl disable firewalld
iptables -F && iptables -X && iptables -F -t nat && iptables -X -t nat && iptables -P FORWARD ACCEPT

4、關(guān)閉SELinux

setenforce 0
sed -i "s/SELINUX=enforcing/SELINUX=disabled/g" /etc/selinux/config

5、關(guān)閉swap分區(qū)

swapoff -a
sed -i '/ swap / s/^\(.*\)$/#\1/g' /etc/fstab

6、設(shè)置內(nèi)核參數(shù)

cat << EOF | tee /etc/sysctl.d/k8s.conf
net.bridge.bridge-nf-call-iptables=1
net.bridge.bridge-nf-call-ip6tables=1
EOF

sysctl -p /etc/sysctl.d/k8s.conf

7、安裝Docker

1、首先卸載舊版：

yum remove docker \
           docker-client \
           docker-client-latest \
           docker-common \
           docker-latest \
           docker-latest-logrotate \
           docker-logrotate \
           docker-selinux \
           docker-engine-selinux \
           docker-engine

2、安裝依賴包：

yum install -y yum-utils device-mapper-persistent-data lvm2

3、設(shè)置安裝源（阿里云）：

yum-config-manager --add-repo http://mirrors.aliyun.com/docker-ce/linux/centos/docker-ce.repo

4、啟用測試庫（可選）：

yum-config-manager --enable docker-ce-edge
yum-config-manager --enable docker-ce-test

5、安裝：

yum makecache fast
yum install -y docker-ce

6、啟動：

systemctl start docker

開機自啟

systemctl enable docker

Docker建議配置阿里云鏡像加速

安裝完成后配置啟動時的命令，否則docker會將iptables FORWARD chain的默認(rèn)策略設(shè)置為DROP，另外Kubeadm建議將systemd設(shè)置為cgroup驅(qū)動，所以還要修改daemon.json

sed -i "13i ExecStartPost=/usr/sbin/iptables -P FORWARD ACCEPT" /usr/lib/systemd/system/docker.service

tee /etc/docker/daemon.json <<-'EOF'
{
  "registry-mirrors": ["https://bk6kzfqm.mirror.aliyuncs.com"],
  "exec-opts": ["native.cgroupdriver=systemd"],
  "log-driver": "json-file",
  "log-opts": {
    "max-size": "100m"
  },
  "storage-driver": "overlay2",
  "storage-opts": [
    "overlay2.override_kernel_check=true"
  ]
}
EOF

systemctl daemon-reload
systemctl restart docker

7、安裝kubeadm和kubelet

配置源

cat <<EOF > /etc/yum.repos.d/kubernetes.repo
[kubernetes]
name=Kubernetes
baseurl=https://mirrors.aliyun.com/kubernetes/yum/repos/kubernetes-el7-x86_64/
enabled=1
gpgcheck=0
repo_gpgcheck=0
gpgkey=https://mirrors.aliyun.com/kubernetes/yum/doc/yum-key.gpg https://mirrors.aliyun.com/kubernetes/yum/doc/rpm-package-key.gpg
EOF

yum makecache fast

安裝

yum install -y kubelet kubeadm kubectl
systemctl enable kubelet

8、拉取所需鏡像

先從阿里云拉取所需的鏡像，不然會從谷歌拉取，導(dǎo)致拉取失敗。

拉取鏡像：

images=(kube-controller-manager-amd64 etcd-amd64 k8s-dns-sidecar-amd64 kube-proxy-amd64 kube-apiserver-amd64 kube-scheduler-amd64 pause-amd64 k8s-dns-dnsmasq-nanny-amd64 k8s-dns-kube-dns-amd64)
for imageName in ${images[@]} ; do
 docker pull champly/$imageName
 docker tag champly/$imageName gcr.io/google_containers/$imageName
 docker rmi champly/$imageName
done

修改版本

docker tag gcr.io/google_containers/etcd-amd64 gcr.io/google_containers/etcd-amd64:3.0.17 && \
docker rmi gcr.io/google_containers/etcd-amd64 && \
docker tag gcr.io/google_containers/k8s-dns-dnsmasq-nanny-amd64 gcr.io/google_containers/k8s-dns-dnsmasq-nanny-amd64:1.14.5 && \
docker rmi gcr.io/google_containers/k8s-dns-dnsmasq-nanny-amd64 && \
docker tag gcr.io/google_containers/k8s-dns-kube-dns-amd64 gcr.io/google_containers/k8s-dns-kube-dns-amd64:1.14.5 && \
docker rmi gcr.io/google_containers/k8s-dns-kube-dns-amd64 && \
docker tag gcr.io/google_containers/k8s-dns-sidecar-amd64 gcr.io/google_containers/k8s-dns-sidecar-amd64:1.14.2 && \
docker rmi gcr.io/google_containers/k8s-dns-sidecar-amd64 && \
docker tag gcr.io/google_containers/kube-apiserver-amd64 gcr.io/google_containers/kube-apiserver-amd64:v1.7.5 && \
docker rmi gcr.io/google_containers/kube-apiserver-amd64 && \
docker tag gcr.io/google_containers/kube-controller-manager-amd64 gcr.io/google_containers/kube-controller-manager-amd64:v1.7.5 && \
docker rmi gcr.io/google_containers/kube-controller-manager-amd64 && \
docker tag gcr.io/google_containers/kube-proxy-amd64 gcr.io/google_containers/kube-proxy-amd64:v1.6.0 && \
docker rmi gcr.io/google_containers/kube-proxy-amd64 && \
docker tag gcr.io/google_containers/kube-scheduler-amd64 gcr.io/google_containers/kube-scheduler-amd64:v1.7.5 && \
docker rmi gcr.io/google_containers/kube-scheduler-amd64 && \
docker tag gcr.io/google_containers/pause-amd64 gcr.io/google_containers/pause-amd64:3.0 && \
docker rmi gcr.io/google_containers/pause-amd64

三、初始化集群

以下命令如無特殊說明，均在node11上執(zhí)行

1、使用kubeadm init初始化集群（注意修改最后為本機IP）

kubeadm init \
  --kubernetes-version=v1.14.3 \
  --pod-network-cidr=10.244.0.0/16 \
  --apiserver-advertise-address=192.168.11.11 

初始化成功后會輸出類似下面的加入命令，暫時無需運行，先記錄。

kubeadm join 192.168.11.11:6443 --token 4kkgk2.ht4hnyeinhk6pwod \
    --discovery-token-ca-cert-hash sha256:8a3d03e783e82b1a066957e3311dd94fa2b76372b9c2b0bc3597a5e357ea5ca9 

2、為需要使用kubectl的用戶進(jìn)行配置

mkdir -p $HOME/.kube
sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
sudo chown $(id -u):$(id -g) $HOME/.kube/config

檢查集群狀態(tài)

kubectl get cs

3、安裝Pod Network（使用七牛云鏡像）

curl -o kube-flannel.yml https://raw.githubusercontent.com/coreos/flannel/master/Documentation/kube-flannel.yml
sed -i "s/quay.io\/coreos\/flannel/quay-mirror.qiniu.com\/coreos\/flannel/g" kube-flannel.yml
kubectl apply -f kube-flannel.yml
rm -f kube-flannel.yml

使用下面的命令確保所有的Pod都處于Running狀態(tài)，可能要等到許久。

kubectl get pod --all-namespaces -o wide

4、向Kubernetes集群中添加Node節(jié)點

在node12和node13上運行之前在node11輸出的命令

kubeadm join 192.168.11.11:6443 --token 4kkgk2.ht4hnyeinhk6pwod \
    --discovery-token-ca-cert-hash sha256:8a3d03e783e82b1a066957e3311dd94fa2b76372b9c2b0bc3597a5e357ea5ca9 

查看集群中的節(jié)點狀態(tài)，可能要等等許久才Ready

kubectl get nodes

5、kube-proxy開啟ipvs

kubectl get configmap kube-proxy -n kube-system -o yaml > kube-proxy-configmap.yaml
sed -i 's/mode: ""/mode: "ipvs"/' kube-proxy-configmap.yaml
kubectl apply -f kube-proxy-configmap.yaml
rm -f kube-proxy-configmap.yaml
kubectl get pod -n kube-system | grep kube-proxy | awk '{system("kubectl delete pod "$1" -n kube-system")}' 

四、部署kubernetes-dashboard

1、生成訪問證書

grep 'client-certificate-data' ~/.kube/config | head -n 1 | awk '{print $2}' | base64 -d >> kubecfg.crt
grep 'client-key-data' ~/.kube/config | head -n 1 | awk '{print $2}' | base64 -d >> kubecfg.key
openssl pkcs12 -export -clcerts -inkey kubecfg.key -in kubecfg.crt -out kubecfg.p12 -name "kubernetes-client"

將生成的kubecfg.p12證書導(dǎo)入到Windows中，直接雙擊打開，下一步導(dǎo)入即可。

注意：導(dǎo)入完成后需重啟瀏覽器。

2、生成訪問Token

新建文件admin-user.yaml：

apiVersion: v1
kind: ServiceAccount
metadata:
  name: admin-user
  namespace: kube-system

---

apiVersion: rbac.authorization.k8s.io/v1beta1
kind: ClusterRoleBinding
metadata:
  name: admin-user
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: cluster-admin
subjects:
- kind: ServiceAccount
  name: admin-user
  namespace: kube-system

創(chuàng)建角色及綁定賬號

kubectl create -f admin-user.yaml

獲取Token

kubectl describe secret admin-user --namespace=kube-system
    eyJhbGciOiJSUzI1NiIsImtpZCI6IiJ9.eyJpc3MiOiJrdWJlcm5ldGVzL3NlcnZpY2VhY2NvdW50Iiwia3ViZXJuZXRlcy5pby9zZXJ2aWNlYWNjb3VudC9uYW1lc3BhY2UiOiJrdWJlLXN5c3RlbSIsImt1YmVybmV0ZXMuaW8vc2VydmljZWFjY291bnQvc2VjcmV0Lm5hbWUiOiJhZG1pbi11c2VyLXRva2VuLWhydmxrIiwia3ViZXJuZXRlcy5pby9zZXJ2aWNlYWNjb3VudC9zZXJ2aWNlLWFjY291bnQubmFtZSI6ImFkbWluLXVzZXIiLCJrdWJlcm5ldGVzLmlvL3NlcnZpY2VhY2NvdW50L3NlcnZpY2UtYWNjb3VudC51aWQiOiJmOGE0YzY4NC04Yjc1LTExZTktYjE2ZC0wMDBjMjk5ZGViOWUiLCJzdWIiOiJzeXN0ZW06c2VydmljZWFjY291bnQ6a3ViZS1zeXN0ZW06YWRtaW4tdXNlciJ9.FftzgCCzKiWKNghPDBDqAfBJPwUgHbEJGMv5fEBEq53oV8O3vlGHmZGRqjUYHiye2qhdg084iIRDv-w03b2KroEiMX0nXYN0l73-XlEl6ecU_v7-66xiS9fDPR0JiI6SW_cyL5k16P4qIwBwk1ze99r_R0t2Q8xiplFMVW02u0zM0IG2xtT5AaXqV5uEX3kg6nThloOmxFGbyIPF743D0WEtbNicVI2YYIPM7B8CxnHZ5_9MJ5qLtjVAttomLy30O5VEgweljnaL70tja_M9DlLsBV1O8q01AFZhfBPPaNtuDrPU-OZkVb9isiMYiL92lQLEIGswWlTj-uhmSTQYGA

此次先記錄下生成的Token

3、部署kubernetes-dashboard

curl -o kubernetes-dashboard.yaml https://raw.githubusercontent.com/kubernetes/dashboard/master/aio/deploy/recommended/kubernetes-dashboard.yaml
vi kubernetes-dashboard.yaml
修改image: k8s.gcr.io/kubernetes-dashboard-amd64:v1.10.1為：
image: mirrorgooglecontainers/kubernetes-dashboard-amd64:v1.10.1    125行
kubectl apply -f kubernetes-dashboard.yaml

4、訪問

https://192.168.11.11:6443/api/v1/namespaces/kube-system/services/https:kubernetes-dashboard:/proxy/

192.168.11.11為MasterIP，6443為apiserver-port

然后在登錄選項中選擇令牌登錄，復(fù)制進(jìn)剛剛生成的令牌即可。