kubernetes二進制部署（二）多節(jié)點部署

這是kubernetes二進制部署的第二篇
如果沒有看過前面第一篇的朋友可以看看下面的：
Kubernetes二進制部署（一）單節(jié)點部署

實驗環(huán)境：

負載均衡：
Nginx1:192.168.35.104/24
Nginx2:192.168.35.105/24

Master節(jié)點：
master1:192.168.35.100/24
master2:192.168.35.103/24

Node節(jié)點：
node1:192.168.35.101/24
node2:192.168.35.102/24

master02部署

1、關閉防火墻

[root@localhost ~]# systemctl stop firewalld.service?
[root@localhost ~]# setenforce 0

2、在master1上操作

（1）復制kubernetes目錄到master2

[root@localhost k8s]# scp -r /opt/kubernetes/ root@192.168.35.103:/opt
The authenticity of host '192.168.35.103 (192.168.35.103)' can't be established.
ECDSA key fingerprint is SHA256:VBIpN6lYzzdqZmVfp5cGBOYgDBwYuVkI55sXRAz2C/s.
ECDSA key fingerprint is MD5:fe:66:e7:bd:10:1d:ce:1b:4b:82:9e:e0:99:23:d4:5f.
Are you sure you want to continue connecting (yes/no)? yes
Warning: Permanently added '192.168.35.103' (ECDSA) to the list of known hosts.
root@192.168.35.103's password:?
token.csv ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?100% ? 84 ? ?55.0KB/s ? 00:00 ? ?
kube-apiserver ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? 100% ?934 ? ? 1.0MB/s ? 00:00 ? ?
kube-scheduler ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? 100% ? 94 ? 122.4KB/s ? 00:00 ? ?
kube-controller-manager ? ? ? ? ? ? ? ? ? ? ? ? ? ?100% ?483 ? 393.2KB/s ? 00:00 ? ?
kube-apiserver ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? 100% ?184MB ?91.9MB/s ? 00:02 ? ?
kubectl ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?100% ? 55MB ?72.2MB/s ? 00:00 ? ?
kube-controller-manager ? ? ? ? ? ? ? ? ? ? ? ? ? ?100% ?155MB ?77.3MB/s ? 00:02 ? ?
kube-scheduler ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? 100% ? 55MB ?74.8MB/s ? 00:00 ? ?
ca-key.pem ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? 100% 1679 ? ? 1.5MB/s ? 00:00 ? ?
ca.pem ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? 100% 1359 ? 984.7KB/s ? 00:00 ? ?
server-key.pem ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? 100% 1675 ? ? 1.0MB/s ? 00:00 ? ?
server.pem ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? 100% 1643 ? 338.6KB/s ? 00:00 ? ?

（2）復制master中的三個組件啟動腳本kube-apiserver.service、kube-controller-manager.service、kube-scheduler.service。

[root@localhost k8s]# scp /usr/lib/systemd/system/{kube-apiserver,kube-controller-manager,kube-scheduler}.service root@192.168.35.103:/usr/lib/systemd/system/
root@192.168.35.103's password:?
kube-apiserver.service ? ? ? ? ? ? ? ? ? ? ? ? ? ? 100% ?282 ? 164.1KB/s ? 00:00 ? ?
kube-controller-manager.service ? ? ? ? ? ? ? ? ? ?100% ?317 ? 294.5KB/s ? 00:00 ? ?
kube-scheduler.service ? ? ? ? ? ? ? ? ? ? ? ? ? ? 100% ?281 ? 352.0KB/s ? 00:00 ? ?

（3）拷貝master1上已有的etcd證書給master2使用

特別注意：master2一定要有etcd證書，否則apiserver服務無法啟動

[root@localhost k8s]# scp -r /opt/etcd/ root@192.168.35.103:/opt/
root@192.168.35.103's password:?
etcd ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? 100% ?516 ? 196.4KB/s ? 00:00 ? ?
etcd ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? 100% ? 18MB ?82.4MB/s ? 00:00 ? ?
etcdctl ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?100% ? 15MB ?55.2MB/s ? 00:00 ? ?
ca-key.pem ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? 100% 1679 ? ? 1.0MB/s ? 00:00 ? ?
ca.pem ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? 100% 1265 ? ? 1.1MB/s ? 00:00 ? ?
server-key.pem ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? 100% 1679 ? ? 2.0MB/s ? 00:00 ? ?
server.pem ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? 100% 1338 ? ? 1.6MB/s ? 00:00 ? ?

3、在master2上操作

（1）修改配置文件kube-apiserver中的IP

[root@localhost ~]# vim /opt/kubernetes/cfg/kube-apiserver?

KUBE_APISERVER_OPTS="--logtostderr=true \
--v=4 \
--etcd-servers=https://192.168.35.100:2379,https://192.168.35.101:2379,https://192.168.35.102:2379 \
--bind-address=192.168.35.103 \
--secure-port=6443 \
--advertise-address=192.168.35.103 \
--allow-privileged=true \
--service-cluster-ip-range=10.0.0.0/24 \
--enable-admission-plugins=NamespaceLifecycle,LimitRanger,ServiceAccount,ResourceQuota,NodeRestriction \
--authorization-mode=RBAC,Node \
--kubelet-https=true \
--enable-bootstrap-token-auth \
--token-auth-file=/opt/kubernetes/cfg/token.csv \
--service-node-port-range=30000-50000 \
--tls-cert-file=/opt/kubernetes/ssl/server.pem ?\
--tls-private-key-file=/opt/kubernetes/ssl/server-key.pem \
--client-ca-file=/opt/kubernetes/ssl/ca.pem \
--service-account-key-file=/opt/kubernetes/ssl/ca-key.pem \
--etcd-cafile=/opt/etcd/ssl/ca.pem \
--etcd-certfile=/opt/etcd/ssl/server.pem \
--etcd-keyfile=/opt/etcd/ssl/server-key.pem"

（2）啟動master2中的三個組件服務

[root@localhost ~]# systemctl start kube-apiserver.service?

[root@localhost ~]# systemctl enable kube-apiserver.service?
Created symlink from /etc/systemd/system/multi-user.target.wants/kube-apiserver.service to /usr/lib/systemd/system/kube-apiserver.service.

[root@localhost cfg]# systemctl status kube-apiserver.service
● kube-apiserver.service - Kubernetes API Server
? ?Loaded: loaded (/usr/lib/systemd/system/kube-apiserver.service; enabled; vendor preset: disabled)
? ?Active: active (running) since 六 2020-02-08 14:29:42 CST; 22s ago
? ? ?Docs: https://github.com/kubernetes/kubernetes
?Main PID: 3287 (kube-apiserver)
? ?CGroup: /system.slice/kube-apiserver.service
? ? ? ? ? ?└─3287 /opt/kubernetes/bin/kube-apiserver --logtostderr=true --v=4 --et...

2月 08 14:29:54 localhost.localdomain kube-apiserver[3287]: I0208 14:29:54.583442...
2月 08 14:29:54 localhost.localdomain kube-apiserver[3287]: I0208 14:29:54.585482...
2月 08 14:29:54 localhost.localdomain kube-apiserver[3287]: I0208 14:29:54.587185...
2月 08 14:29:54 localhost.localdomain kube-apiserver[3287]: I0208 14:29:54.588646...
2月 08 14:29:54 localhost.localdomain kube-apiserver[3287]: I0208 14:29:54.590160...
2月 08 14:29:54 localhost.localdomain kube-apiserver[3287]: I0208 14:29:54.591712...
2月 08 14:29:54 localhost.localdomain kube-apiserver[3287]: I0208 14:29:54.593487...
2月 08 14:29:54 localhost.localdomain kube-apiserver[3287]: I0208 14:29:54.598787...
2月 08 14:30:03 localhost.localdomain kube-apiserver[3287]: I0208 14:30:03.331185...
2月 08 14:30:03 localhost.localdomain kube-apiserver[3287]: I0208 14:30:03.333531...
Hint: Some lines were ellipsized, use -l to show in full.

[root@localhost ~]# systemctl start kube-controller-manager.service?

[root@localhost ~]# systemctl enable kube-controller-manager.service?
Created symlink from /etc/systemd/system/multi-user.target.wants/kube-controller-manager.service to /usr/lib/systemd/system/kube-controller-manager.service.

[root@localhost ~]# systemctl start kube-scheduler.service?

[root@localhost ~]# systemctl enable kube-scheduler.service?
Created symlink from /etc/systemd/system/multi-user.target.wants/kube-scheduler.service to /usr/lib/systemd/system/kube-scheduler.service.

（3）增加環(huán)境變量（在配置文件末行加入）

[root@localhost ~]# vim /etc/profile

export PATH=$PATH:/opt/kubernetes/bin/? ? ? ? ? ? ? ? ##添加

[root@localhost ~]# source /etc/profile? ? ? ? ? ?##重新加載

（4）查看群集中的節(jié)點

[root@localhost cfg]# kubectl get node
NAME ? ? ? ? ? ? STATUS ? ROLES ? ?AGE ? VERSION
192.168.35.101 ? Ready ? ?<none> ? 17m ? v1.12.3
192.168.35.102 ? Ready ? ?<none> ? 10m ? v1.12.3