您好,登錄后才能下訂單哦!
這篇文章給大家介紹ElasticSearch如何進(jìn)行角色分離部署,內(nèi)容非常詳細(xì),感興趣的小伙伴們可以參考借鑒,希望對(duì)大家能有所幫助。
ES的架構(gòu)為三節(jié)點(diǎn),即master、ingest、data角色同時(shí)部署在三臺(tái)服務(wù)器上。
下面將進(jìn)行角色分離部署,并且每個(gè)角色分別部署三節(jié)點(diǎn),在實(shí)現(xiàn)性能最大化的同時(shí)保障高可用。
? elasticsearch的master節(jié)點(diǎn):用于調(diào)度,采用普通性能服務(wù)器來(lái)部署
? elasticsearch的ingest節(jié)點(diǎn):用于數(shù)據(jù)預(yù)處理,采用性能好的服務(wù)器來(lái)部署
? elasticsearch的data節(jié)點(diǎn):用于數(shù)據(jù)落地存儲(chǔ),采用存儲(chǔ)性能好的服務(wù)器來(lái)部署
服務(wù)器配置
注意:此處的架構(gòu)是之前的文章《EFK教程 - 快速入門指南》的拓展,因此請(qǐng)先按照《EFK教程 - 快速入門指南》完成部署
1?? 部署3臺(tái)data節(jié)點(diǎn),加入原集群
2?? 部署3臺(tái)ingest節(jié)點(diǎn),加入原集群
3?? 將原有的es索引遷移到data節(jié)點(diǎn)
4?? 將原有的es節(jié)點(diǎn)改造成master節(jié)點(diǎn)
之前已完成了基礎(chǔ)的elasticsearch架構(gòu),現(xiàn)需要新增三臺(tái)存儲(chǔ)節(jié)點(diǎn)加入集群,同時(shí)關(guān)閉master和ingest功能
elasticsearch-data安裝:3臺(tái)均執(zhí)行相同的安裝步驟
tar -zxvf elasticsearch-7.3.2-linux-x86_64.tar.gz mv elasticsearch-7.3.2 /opt/elasticsearch useradd elasticsearch -d /opt/elasticsearch -s /sbin/nologin mkdir -p /opt/logs/elasticsearch chown elasticsearch.elasticsearch /opt/elasticsearch -R chown elasticsearch.elasticsearch /opt/logs/elasticsearch -R # 數(shù)據(jù)盤需要elasticsearch寫權(quán)限 chown elasticsearch.elasticsearch /data/SAS -R # 限制一個(gè)進(jìn)程可以擁有的VMA(虛擬內(nèi)存區(qū)域)的數(shù)量要超過(guò)262144,不然elasticsearch會(huì)報(bào)max virtual memory areas vm.max_map_count [65535] is too low, increase to at least [262144] echo "vm.max_map_count = 655350" >> /etc/sysctl.conf sysctl -p
elasticsearch-data配置
? 192.168.1.51 /opt/elasticsearch/config/elasticsearch.yml
cluster.name: my-application node.name: 192.168.1.51 # 數(shù)據(jù)盤位置,如果有多個(gè)硬盤位置,用","隔開 path.data: /data/SAS path.logs: /opt/logs/elasticsearch network.host: 192.168.1.51 discovery.seed_hosts: ["192.168.1.31","192.168.1.32","192.168.1.33"] cluster.initial_master_nodes: ["192.168.1.31","192.168.1.32","192.168.1.33"] http.cors.enabled: true http.cors.allow-origin: "*" # 關(guān)閉master功能 node.master: false # 關(guān)閉ingest功能 node.ingest: false # 開啟data功能 node.data: true
? 192.168.1.52 /opt/elasticsearch/config/elasticsearch.yml
cluster.name: my-application node.name: 192.168.1.52 # 數(shù)據(jù)盤位置,如果有多個(gè)硬盤位置,用","隔開 path.data: /data/SAS path.logs: /opt/logs/elasticsearch network.host: 192.168.1.52 discovery.seed_hosts: ["192.168.1.31","192.168.1.32","192.168.1.33"] cluster.initial_master_nodes: ["192.168.1.31","192.168.1.32","192.168.1.33"] http.cors.enabled: true http.cors.allow-origin: "*" # 關(guān)閉master功能 node.master: false # 關(guān)閉ingest功能 node.ingest: false # 開啟data功能 node.data: true
? 192.168.1.53 /opt/elasticsearch/config/elasticsearch.yml
cluster.name: my-application node.name: 192.168.1.53 # 數(shù)據(jù)盤位置,如果有多個(gè)硬盤位置,用","隔開 path.data: /data/SAS path.logs: /opt/logs/elasticsearch network.host: 192.168.1.53 discovery.seed_hosts: ["192.168.1.31","192.168.1.32","192.168.1.33"] cluster.initial_master_nodes: ["192.168.1.31","192.168.1.32","192.168.1.33"] http.cors.enabled: true http.cors.allow-origin: "*" # 關(guān)閉master功能 node.master: false # 關(guān)閉ingest功能 node.ingest: false # 開啟data功能 node.data: true
elasticsearch-data啟動(dòng)
sudo -u elasticsearch /opt/elasticsearch/bin/elasticsearch
elasticsearch集群狀態(tài)
curl "http://192.168.1.31:9200/_cat/health?v"
elasticsearch-data狀態(tài)
curl "http://192.168.1.31:9200/_cat/nodes?v"
elasticsearch-data參數(shù)說(shuō)明
status: green # 集群健康狀態(tài) node.total: 6 # 有6臺(tái)機(jī)子組成集群 node.data: 6 # 有6個(gè)節(jié)點(diǎn)的存儲(chǔ) node.role: d # 只擁有data角色 node.role: i # 只擁有ingest角色 node.role: m # 只擁有master角色 node.role: mid # 擁master、ingest、data角色
現(xiàn)需要新增三臺(tái)ingest節(jié)點(diǎn)加入集群,同時(shí)關(guān)閉master和data功能
elasticsearch-ingest安裝:3臺(tái)es均執(zhí)行相同的安裝步驟
tar -zxvf elasticsearch-7.3.2-linux-x86_64.tar.gz mv elasticsearch-7.3.2 /opt/elasticsearch useradd elasticsearch -d /opt/elasticsearch -s /sbin/nologin mkdir -p /opt/logs/elasticsearch chown elasticsearch.elasticsearch /opt/elasticsearch -R chown elasticsearch.elasticsearch /opt/logs/elasticsearch -R # 限制一個(gè)進(jìn)程可以擁有的VMA(虛擬內(nèi)存區(qū)域)的數(shù)量要超過(guò)262144,不然elasticsearch會(huì)報(bào)max virtual memory areas vm.max_map_count [65535] is too low, increase to at least [262144] echo "vm.max_map_count = 655350" >> /etc/sysctl.conf sysctl -p
elasticsearch-ingest配置
? 192.168.1.41 /opt/elasticsearch/config/elasticsearch.yml
cluster.name: my-application node.name: 192.168.1.41 path.logs: /opt/logs/elasticsearch network.host: 192.168.1.41 discovery.seed_hosts: ["192.168.1.31","192.168.1.32","192.168.1.33"] cluster.initial_master_nodes: ["192.168.1.31","192.168.1.32","192.168.1.33"] http.cors.enabled: true http.cors.allow-origin: "*" # 關(guān)閉master功能 node.master: false # 開啟ingest功能 node.ingest: true # 關(guān)閉data功能 node.data: false
? 192.168.1.42 /opt/elasticsearch/config/elasticsearch.yml
cluster.name: my-application node.name: 192.168.1.42 path.logs: /opt/logs/elasticsearch network.host: 192.168.1.42 discovery.seed_hosts: ["192.168.1.31","192.168.1.32","192.168.1.33"] cluster.initial_master_nodes: ["192.168.1.31","192.168.1.32","192.168.1.33"] http.cors.enabled: true http.cors.allow-origin: "*" # 關(guān)閉master功能 node.master: false # 開啟ingest功能 node.ingest: true # 關(guān)閉data功能 node.data: false
? 192.168.1.43 /opt/elasticsearch/config/elasticsearch.yml
cluster.name: my-application node.name: 192.168.1.43 path.logs: /opt/logs/elasticsearch network.host: 192.168.1.43 discovery.seed_hosts: ["192.168.1.31","192.168.1.32","192.168.1.33"] cluster.initial_master_nodes: ["192.168.1.31","192.168.1.32","192.168.1.33"] http.cors.enabled: true http.cors.allow-origin: "*" # 關(guān)閉master功能 node.master: false # 開啟ingest功能 node.ingest: true # 關(guān)閉data功能 node.data: false
elasticsearch-ingest啟動(dòng)
sudo -u elasticsearch /opt/elasticsearch/bin/elasticsearch
elasticsearch集群狀態(tài)
curl "http://192.168.1.31:9200/_cat/health?v"
elasticsearch-ingest狀態(tài)
curl "http://192.168.1.31:9200/_cat/nodes?v"
elasticsearch-ingest參數(shù)說(shuō)明
status: green # 集群健康狀態(tài) node.total: 9 # 有9臺(tái)機(jī)子組成集群 node.data: 6 # 有6個(gè)節(jié)點(diǎn)的存儲(chǔ) node.role: d # 只擁有data角色 node.role: i # 只擁有ingest角色 node.role: m # 只擁有master角色 node.role: mid # 擁master、ingest、data角色
首先,將上一篇《EFK教程 - 快速入門指南》中部署的3臺(tái)es(192.168.1.31、192.168.1.32、192.168.1.33)改成只有master的功能, 因此需要先將這3臺(tái)上的索引數(shù)據(jù)遷移到本次所做的data節(jié)點(diǎn)中
1?? 索引遷移:一定要做這步,將之前的索引放到data節(jié)點(diǎn)上
curl -X PUT "192.168.1.31:9200/*/_settings?pretty" -H 'Content-Type: application/json' -d' { "index.routing.allocation.include._ip": "192.168.1.51,192.168.1.52,192.168.1.53" }'
2?? 確認(rèn)當(dāng)前索引存儲(chǔ)位置:確認(rèn)所有索引不在192.168.1.31、192.168.1.32、192.168.1.33節(jié)點(diǎn)上
curl "http://192.168.1.31:9200/_cat/shards?h=n"
elasticsearch-master配置
注意事項(xiàng):修改配置,重啟進(jìn)程,需要一臺(tái)一臺(tái)執(zhí)行,要確保第一臺(tái)成功后,再執(zhí)行下一臺(tái)。重啟進(jìn)程的方法:由于上一篇文章《EFK教程 - 快速入門指南》里,是執(zhí)行命令跑在前臺(tái),因此直接ctrl - c退出再啟動(dòng)即可,啟動(dòng)命令如下
sudo -u elasticsearch /opt/elasticsearch/bin/elasticsearch
? 192.168.1.31 /opt/elasticsearch/config/elasticsearch.yml
cluster.name: my-application node.name: 192.168.1.31 path.logs: /opt/logs/elasticsearch network.host: 192.168.1.31 discovery.seed_hosts: ["192.168.1.31","192.168.1.32","192.168.1.33"] cluster.initial_master_nodes: ["192.168.1.31","192.168.1.32","192.168.1.33"] http.cors.enabled: true http.cors.allow-origin: "*" #開啟master功能 node.master: true #關(guān)閉ingest功能 node.ingest: false #關(guān)閉data功能 node.data: false
? 192.168.1.32 /opt/elasticsearch/config/elasticsearch.yml
cluster.name: my-application node.name: 192.168.1.32 path.logs: /opt/logs/elasticsearch network.host: 192.168.1.32 discovery.seed_hosts: ["192.168.1.31","192.168.1.32","192.168.1.33"] cluster.initial_master_nodes: ["192.168.1.31","192.168.1.32","192.168.1.33"] http.cors.enabled: true http.cors.allow-origin: "*" #開啟master功能 node.master: true #關(guān)閉ingest功能 node.ingest: false #關(guān)閉data功能 node.data: false
? 192.168.1.33 /opt/elasticsearch/config/elasticsearch.yml
cluster.name: my-application node.name: 192.168.1.33 path.logs: /opt/logs/elasticsearch network.host: 192.168.1.33 discovery.seed_hosts: ["192.168.1.31","192.168.1.32","192.168.1.33"] cluster.initial_master_nodes: ["192.168.1.31","192.168.1.32","192.168.1.33"] http.cors.enabled: true http.cors.allow-origin: "*" #開啟master功能 node.master: true #關(guān)閉ingest功能 node.ingest: false #關(guān)閉data功能 node.data: false
elasticsearch集群狀態(tài)
curl "http://192.168.1.31:9200/_cat/health?v"
elasticsearch-master狀態(tài)
curl "http://192.168.1.31:9200/_cat/nodes?v"
至此,當(dāng)node.role里所有服務(wù)器都不再出現(xiàn)“mid”,則表示一切順利完成。
關(guān)于ElasticSearch如何進(jìn)行角色分離部署就分享到這里了,希望以上內(nèi)容可以對(duì)大家有一定的幫助,可以學(xué)到更多知識(shí)。如果覺(jué)得文章不錯(cuò),可以把它分享出去讓更多的人看到。
免責(zé)聲明:本站發(fā)布的內(nèi)容(圖片、視頻和文字)以原創(chuàng)、轉(zhuǎn)載和分享為主,文章觀點(diǎn)不代表本網(wǎng)站立場(chǎng),如果涉及侵權(quán)請(qǐng)聯(lián)系站長(zhǎng)郵箱:is@yisu.com進(jìn)行舉報(bào),并提供相關(guān)證據(jù),一經(jīng)查實(shí),將立刻刪除涉嫌侵權(quán)內(nèi)容。