分布式鍵值高可用存儲集群架構(gòu)搭建（ETCD)

   ETCD 是一個高可用的分布式鍵值數(shù)據(jù)庫，可用于服務(wù)發(fā)現(xiàn)。ETCD 采用 raft 一致性算法，基于 Go 語言實現(xiàn)，隨著CoreOS和Kubernetes等項目在開源社區(qū)日益火熱，它們項目中都用到的etcd組件作為一個高可用強一致性的服務(wù)發(fā)現(xiàn)存儲倉庫，漸漸為開發(fā)人員所關(guān)注。在云計算時代，如何讓服務(wù)快速透明地接入到計算集群中，如何讓共享配置信息快速被集群中的所有機器發(fā)現(xiàn)，更為重要的是，如何構(gòu)建這樣一套高可用、安全、易于部署以及響應(yīng)快速的服務(wù)集群，已經(jīng)成為了迫切需要解決的問題。etcd為解決這類問題帶來了福音，本文將從etcd的應(yīng)用場景開始，深入解讀etcd的實現(xiàn)方式，以供開發(fā)者們更為充分地享用etcd所帶來的便利。

特點
簡單：安裝配置使用簡單，提供 HTTP API
安全：支持 SSL 證書
可靠：采用 raft 算法，實現(xiàn)分布式系統(tǒng)數(shù)據(jù)的可用性和一致性
下載安裝包：
[root@master ~]# wget https://github.com/coreos/etcd/releases/download/v3.3.2/etcd-v3.3.2-linux-amd64.tar.gz

復(fù)制安裝包到各接點：
[root@master ~]# ansible k8s -m copy -a'src=/root/etcd-v3.3.2-linux-amd64.tar.gz dest=/root/'

部署環(huán)境三臺機子：IP--主機名--集群結(jié)點名
192.168.27.211 client1 (etcd1)
192.168.27.212 client2 (etcd2)
192.168.27.213 client3 (etcd3)

1. 解壓版本包
   tar -zxvf etcd-v3.3.2-linux-amd64.tar.gz 或 [root@master ~]# ansible k8s -m shell -a'tar -zxf etcd-v3.3.2-linux-amd64.tar.gz'
   
   
2. 將解壓文件重命名并移到/etc/目錄下
   mv etcd-v3.3.2-linux-amd64 /etc/etcd-v3.3.2 或
   [root@master ~]# ansible k8s -m shell -a'mv etcd-v3.3.2-linux-amd64 /etc/etcd-v3.3.2'
   
3. 將etcd和etcdctl加入系統(tǒng)變量PATH中(可以省略)
   [root@master ~]# ansible k8s -m shell -a'echo export PATH=/etc/etcd-v3.3.2:$PATH >> ~/.bashrc'
   [root@master ~]# ansible k8s -m shell -a'source ~/.bashrc '
   [root@master ~]# ansible k8s -m shell -a'source /root/.bashrc ' #靠普些
   
   
   4.進入/etc/etcd-v3.3.2，并創(chuàng)建配置文件etcd.conf，內(nèi)容：
   
   
   name: etcd1
   data-dir: /etc/etcd-v3.3.2/data
   listen-client-urls: http://0.0.0.0:2379
   advertise-client-urls: http://192.168.27.211:2379
   listen-peer-urls: http://0.0.0.0:2380
   initial-advertise-peer-urls: http://192.168.27.211:2380
   initial-cluster: etcd1=http://192.168.27.211:2380,etcd2=http://192.168.27.212:2380,etcd3=http://192.168.27.213:2380
   initial-cluster-token: etcd-cluster-my
   initial-cluster-state: new
   5.設(shè)置etcd版本號，默認為2，我們安裝的版本為3.3.2，所以此處設(shè)置為3
   
   在 文件~/.bashrc中加入一行改為3
   export ETCDCTL_API=3
   
   
   
4. 設(shè)置開機啟動，新建文件/usr/lib/systemd/system/etcd.service，設(shè)置方便管理配置內(nèi)容：
   [root@client1 etcd-v3.3.2]# cat /usr/lib/systemd/system/etcd.service
   [Unit]
   Description=Etcd Server
   After=network.target
   After=network-online.target
   Wants=network-online.target

[Service]
Type=notify
WorkingDirectory=/etc/etcd-v3.3.2
#User=etcd
ExecStart=/etc/etcd-v3.3.2/etcd --config-file /etc/etcd-v3.3.2/etcd.conf
Restart=on-failure
LimitNOFILE=65536

[Install]
WantedBy=multi-user.target

5. 啟動etcd，啟動第一個member的時候，此member等待其它member初始化完畢，處于watting狀態(tài)，此時緊接著啟動第二個就行了
   systemctl daemon-reload
   systemctl enable etcd
   systemctl start etcd
   systemctl restart etcd

注意：192.168.27.212，192.168.27.213部署如同上面步驟，只需要將步驟4里面接點名字、ip改為自己ip即可。

name: etcd2
data-dir: /etc/etcd-v3.3.2/data
listen-client-urls: http://0.0.0.0:2379
advertise-client-urls: http://192.168.27.212:2379
listen-peer-urls: http://0.0.0.0:2380
initial-advertise-peer-urls: http://192.168.27.212:2380
initial-cluster: etcd1=http://192.168.27.211:2380,etcd2=http://192.168.27.212:2380,etcd3=http://192.168.27.213:2380
initial-cluster-token: etcd-cluster-my
initial-cluster-state: new

name: etcd3
data-dir: /etc/etcd-v3.3.2/data
listen-client-urls: http://0.0.0.0:2379
advertise-client-urls: http://192.168.27.213:2379
listen-peer-urls: http://0.0.0.0:2380
initial-advertise-peer-urls: http://192.168.27.213:2380
initial-cluster: etcd1=http://192.168.27.211:2380,etcd2=http://192.168.27.212:2380,etcd3=http://192.168.27.213:2380
initial-cluster-token: etcd-cluster-my
initial-cluster-state: new
6、啟動成功后查看集群成員及狀態(tài):
[root@client3 etcd-v3.3.2]# etcdctl member list

etcdctl endpoint health --endpoints=192.168.27.211:2379,192.168.27.212:2379,192.168.27.213:2379

[root@client3 etcd-v3.3.2]# etcdctl endpoint status --endpoints=192.168.27.211:2379,192.168.27.212:2379,192.168.27.213:2379

[root@client3 etcd-v3.3.2]# etcdctl endpoint status --endpoints=192.168.27.211:2379

通過非ETCD集群成員訪問測試：
[root@master etcd-v3.3.2]# etcdctl endpoint status --endpoints=192.168.27.211:2379
192.168.27.211:2379, f4e909b85dbd820b, 3.3.2, 25 kB, false, 69, 71
[root@master etcd-v3.3.2]# etcdctl endpoint status --endpoints=192.168.27.211:2379
192.168.27.211:2379, f4e909b85dbd820b, 3.3.2, 25 kB, false, 69, 71
[root@master etcd-v3.3.2]# etcdctl endpoint status --endpoints=192.168.27.211:2379,192.168.27.212:2379,192.168.27.213:2379
192.168.27.211:2379, f4e909b85dbd820b, 3.3.2, 25 kB, false, 69, 71
192.168.27.212:2379, 265ebeb3298fa51f, 3.3.2, 25 kB, false, 69, 71
192.168.27.213:2379, 510ff85ed64c7436, 3.3.2, 25 kB, true, 69, 71
[root@master etcd-v3.3.2]# etcdctl put name jerry --endpoints=192.168.27.211:2379,192.168.27.212:2379,192.168.27.213:2379
OK
[root@master etcd-v3.3.2]# etcdctl put topic "this is a etcd client test" --endpoints=192.168.27.211:2379,192.168.27.212:2379,192.168.27.213:2379
OK
[root@master etcd-v3.3.2]# etcdctl get topic --endpoints=192.168.27.211:2379,192.168.27.212:2379,192.168.27.213:2379
topic
this is a etcd client test
[root@master etcd-v3.3.2]# etcdctl get topic --endpoints=192.168.27.211:2379
topic
this is a etcd client test

Etcd使用：
讀寫鍵值：
[root@client1 etcd-v3.3.2]# etcdctl put key1 value1
OK
[root@client1 etcd-v3.3.2]# etcdctl get key1
key1
value1

前綴讀（將所有前綴是key的鍵值對都讀出來）

鍵值刪除：
[root@client1 etcd-v3.3.2]# etcdctl del key
0
[root@client1 etcd-v3.3.2]# etcdctl get key
[root@client1 etcd-v3.3.2]# etcdctl get key1
key1
value1
[root@client1 etcd-v3.3.2]# etcdctl del key1
1
[root@client1 etcd-v3.3.2]# etcdctl get key1

觀察已無KEY1鍵及值的存在。

按前綴全部刪除

觀察已無鍵值及數(shù)據(jù)

重新創(chuàng)建三個鍵值：

其它ETCD集群成員上訪問鍵值測試：

監(jiān)聽某個鍵，每次鍵值有變化
[root@client1 etcd-v3.3.2]# etcdctl watch key --prefix
PUT
key1
value
PUT
key2
jerry

設(shè)置過期時間TTL


[root@client2 etcd-v3.3.2]# etcdctl lease grant 30
lease 251f6cf538287411 granted with TTL(30s)
[root@client2 etcd-v3.3.2]# etcdctl put key5 value5 --lease=251f6cf538287411
Error: etcdserver: requested lease not found
[root@client2 etcd-v3.3.2]# etcdctl lease grant 30
lease 251f6cf538287414 granted with TTL(30s)
[root@client2 etcd-v3.3.2]# etcdctl put key5 value5 --lease=251f6cf538287414
OK
[root@client2 etcd-v3.3.2]# etcdctl lease keep-alive 251f6cf538287414
lease 251f6cf538287414 expired or revoked.
[root@client2 etcd-v3.3.2]# etcdctl get key5
[root@client2 etcd-v3.3.2]# etcdctl get key5