ETCD集群運(yùn)維—備份及容災(zāi)恢復(fù)

etcd是一個(gè)開源的分布式鍵值對(duì)數(shù)據(jù)庫，他的每一個(gè)節(jié)點(diǎn)都有一份數(shù)據(jù)的copy，當(dāng)有節(jié)點(diǎn)故障時(shí)保證了高可用性。etcd使用Raft算法來保證一致性。

特別提醒：若使用v3備份數(shù)據(jù)時(shí)存在v2的數(shù)據(jù)則不影響恢復(fù)

若使用v2備份數(shù)據(jù)時(shí)存在v3的數(shù)據(jù)則恢復(fù)失敗

etcd V2新增節(jié)點(diǎn)(刪除數(shù)據(jù)后重新加入)

適用于單節(jié)點(diǎn)故障，重新加入集群

V2與V3新增節(jié)點(diǎn)的區(qū)別僅在于member add參數(shù)

停止etcd服務(wù)

# systemctl stop etcd

# 清空etcd data數(shù)據(jù)

# mv /var/lib/etcd/default.etcd /tmp

# 查看etcd列表

# /opt/kubernetes/bin/etcdctl --ca-file=ca.pem --cert-file=server.pem --key-file=server-key.pem --endpoints="https://172.30.0.89:2379,https://172.30.0.81:2379,https://172.30.2.10:2379" member list

# remove 已存在的etcd name

# /opt/kubernetes/bin/etcdctl --ca-file=ca.pem --cert-file=server.pem --key-file=server-key.pem --endpoints="https://172.30.0.89:2379,https://172.30.0.81:2379,https://172.30.2.10:2379" member remove a4ec138b3529c929

# 新增etcd節(jié)點(diǎn)

# V2

# /opt/kubernetes/bin/etcdctl --ca-file=/opt/kubernetes/ssl/ca.pem --cert-file=/opt/kubernetes/ssl/server.pem --key-file=/opt/kubernetes/ssl/server-key.pem member add etcd03 https://172.30.0.89:2380

# V3

# ETCDCTL_API=3? /opt/kubernetes/bin/etcdctl --endpoints=https://172.30.2.10:2379,etcd02=https://172.30.0.81:2379,etcd03=https://172.30.0.89:2379? --cacert=/opt/kubernetes/ssl/ca.pem? ?--cert=/opt/kubernetes/ssl/server.pem? ?--key=/opt/kubernetes/ssl/server-key.pem? member add? etcd03 --peer-urls="https://172.30.0.89:2380"?

# 修改該節(jié)點(diǎn)etcd配置ETCD_INITIAL_CLUSTER_STATE="existing"

# 同時(shí)重啟所有etcd

# systemctl restart etcd

etcd數(shù)據(jù)備份恢復(fù)（集群不可用，災(zāi)難恢復(fù)）

注意：單節(jié)點(diǎn)備份一份當(dāng)前或歷史數(shù)據(jù)即可，在恢復(fù)的時(shí)候需要同步到所有節(jié)點(diǎn)，若只想恢復(fù)一臺(tái)節(jié)點(diǎn)的，則會(huì)在系統(tǒng)日志里報(bào)request cluster ID mismatch

沒有節(jié)點(diǎn)加入，因?yàn)槭菚?huì)創(chuàng)建一個(gè)新的集群id ，相當(dāng)于重新搭建一臺(tái)新的cluster（數(shù)據(jù)保留，cluster id變更）

# 停止apiserver（集群不可用狀態(tài)，確認(rèn)無寫入的話也可以不停）

# 停止etcd服務(wù)

systemctl stop etcd

# 數(shù)據(jù)備份

# ETCDCTL_API=3 /opt/kubernetes/bin/etcdctl snapshot? save /var/lib/etcd/etcd_backup/etcd_$(date "+%Y%m%d%H%M%S").db

# 同步備份數(shù)據(jù)到各個(gè)節(jié)點(diǎn)

# scp etcd_$(date "+%Y%m%d%H%M%S").db? 集群各etcd節(jié)點(diǎn)

# 移除數(shù)據(jù)目錄?

mv /var/lib/etcd/default.etcd /tmp

# 快照恢復(fù)數(shù)據(jù)? 修改name 和 initial-advertise-peer-urls為當(dāng)前節(jié)點(diǎn)的即可

# ETCDCTL_API=3 /opt/kubernetes/bin/etcdctl snapshot --cacert=/opt/kubernetes/ssl/ca.pem --cert=/opt/kubernetes/ssl/server.pem --key=/opt/kubernetes/ssl/server-key.pem? restore /var/lib/etcd/etcd_20190903102702.db --name=etcd01 --data-dir=/var/lib/etcd/default.etcd --initial-cluster="etcd01=https://172.30.2.10:2380,etcd02=https://172.30.0.81:2380,etcd03=https://172.30.0.89:2380" --initial-cluster-token="etcd-cluster" --initial-advertise-peer-urls=https://172.30.2.10:2380

# 重啟etcd