您好,登錄后才能下訂單哦!
一、概要
本文記錄hbase高可用集群部署過(guò)程,在部署hbase之前需要事先部署好hadoop集群,因?yàn)閔base的數(shù)據(jù)需要存放在hdfs上,hadoop集群的部署后續(xù)會(huì)有一篇文章記錄,本文假設(shè)hadoop集群已經(jīng)部署好,分布式hbase集群需要依賴zk,并且zk可以是hbase自己托管的也可以是我們自己?jiǎn)为?dú)搭建的,這里我們使用自己?jiǎn)为?dú)搭建的zk集群,我們的hadoop集群是用的cdh的發(fā)行版,所以hbase也會(huì)使用cdh的源。
二、環(huán)境
1、軟件版本
centos6
zookeeper-3.4.5+cdh6.9.0+98-1.cdh6.9.0.p0.30.el6.x86_64
hadoop-2.6.0+cdh6.9.0+1799-1.cdh6.9.0.p0.30.el6.x86_64
hbase-1.2.0+cdh6.9.0+205-1.cdh6.9.0.p0.30.el6.x86_64
2、角色
a、zk集群
10.10.20.64:2181 10.10.40.212:2181 10.10.102.207:2181
b、hbase
10.10.40.212 HMaster 10.10.20.64 HMaster 10.10.10.114 HRegionServer 10.10.40.169 HRegionServer 10.10.30.174 HRegionServer
三、部署
1、配置cdh的yum源
vim /etc/yum.repos.d/cloudera-cdh.repo [cloudera-cdh6] # Packages for Cloudera's Distribution for Hadoop, Version 5.4.4, on RedHat or CentOS 6 x86_64 name=Cloudera's Distribution for Hadoop, Version 5.4.8 baseurl=http://archive.cloudera.com/cdh6/redhat/6/x86_64/cdh/5.9.0/ gpgkey=http://archive.cloudera.com/cdh6/redhat/6/x86_64/cdh/RPM-GPG-KEY-cloudera gpgcheck=1 [cloudera-gplextras5b2] # Packages for Cloudera's GPLExtras, Version 5.4.4, on RedHat or CentOS 6 x86_64 name=Cloudera's GPLExtras, Version 5.4.8 baseurl=http://archive.cloudera.com/gplextras5/redhat/6/x86_64/gplextras/5.9.0/ gpgkey=http://archive.cloudera.com/gplextras5/redhat/6/x86_64/gplextras/RPM-GPG-KEY-cloudera gpgcheck=1
2、安裝zk集群(所有zk節(jié)點(diǎn)都操作)
1、安裝
yum -y install zookeeper zookeeper-server
b、配置
vim /etc/zookeeper/conf/zoo.cfg tickTime=2000 initLimit=10 syncLimit=5 dataDir=/data/lib/zookeeper clientPort=2181 maxClientCnxns=0 server.1=10.10.20.64:2888:3888 server.2=10.10.40.212:2888:3888 server.3=10.10.102.207:2888:3888 autopurge.snapRetainCount=3 autopurge.purgeInterval=1
mkdir -p /data/lib/zookeeper #建zk的dir目錄
echo 1 >/data/lib/zookeeper/myid #10.10.20.64上操作 echo 2 >/data/lib/zookeeper/myid #10.10.40.212上操作 echo 3 >/data/lib/zookeeper/myid #10.10.102.207上操作
c、啟動(dòng)服務(wù)
/etc/init.d/zookeeper-server start
3、安裝配置hbase集群
a、安裝
yum -y install hbase hbase-master #HMaster節(jié)點(diǎn)操作 yum -y install hbase hbase-regionserver #HRegionServer節(jié)點(diǎn)操作
b、配置(所有base節(jié)點(diǎn)操作)
vim /etc/hbase/conf/hbase-site.xml <?xml version="1.0"?> <?xml-stylesheet type="text/xsl" href="configuration.xsl"?> <configuration> <property> <name>hbase.zookeeper.quorum</name> <value>10.10.20.64:2181,10.10.40.212:2181,10.10.102.207:2181</value> </property> <property> <name>hbase.zookeeper.property.clientPort</name> <value>2181</value> </property> <property> <name>hbase.zookeeper.property.dataDir</name> <value>/data/lib/zookeeper/</value> </property> <property> <name>hbase.rootdir</name> <value>hdfs://mycluster:8020/hbase</value> </property> <property> <name>hbase.cluster.distributed</name> <value>true</value> <description>集群的模式,分布式還是單機(jī)模式,如果設(shè)置成false的話,HBase進(jìn)程和Zookeeper進(jìn)程在同一個(gè)JVM進(jìn)程 </description> </property> </configuration>
echo "export HBASE_MANAGES_ZK=false" >>/etc/hbase/conf/hbase-env.sh #設(shè)置hbase使用獨(dú)立的zk集群
vim /etc/hbase/conf/regionservers ip-10-10-30-174.ec2.internal ip-10-10-10-114.ec2.internal ip-10-10-40-169.ec2.internal #添加HRegionServer的主機(jī)名到regionservers,我沒(méi)有在/etc/hosts下做主機(jī)名的映射,直接用了ec2的默認(rèn)主機(jī)名
c、啟動(dòng)服務(wù)
/etc/init.d/hbase-master start #HMaster節(jié)點(diǎn)操作 /etc/init.d/hbase-regionserver start #HRegionServer節(jié)點(diǎn)操作
4、驗(yàn)證
a、驗(yàn)證基本功能
[root@ip-10-10-20-64 ~]# hbase shell 2017-05-10 16:31:20,225 INFO [main] Configuration.deprecation: hadoop.native.lib is deprecated. Instead, use io.native.lib.available HBase Shell; enter 'help<RETURN>' for list of supported commands. Type "exit<RETURN>" to leave the HBase Shell Version 1.2.0-cdh6.9.0, rUnknown, Fri Oct 21 01:19:47 PDT 2016 hbase(main):001:0> status 1 active master, 1 backup masters, 3 servers, 0 dead, 1.3333 average load hbase(main):002:0> list TABLE test test1 2 row(s) in 0.0330 seconds => ["test", "test1"] hbase(main):003:0> describe 'test' Table test is ENABLED test COLUMN FAMILIES DESCRIPTION {NAME => 'id', BLOOMFILTER => 'ROW', VERSIONS => '1', IN_MEMORY => 'false', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', COMPRESSION => 'NONE', MIN_VERSIO NS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} {NAME => 'name', BLOOMFILTER => 'ROW', VERSIONS => '1', IN_MEMORY => 'false', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', COMPRESSION => 'NONE', MIN_VERS IONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} {NAME => 'text', BLOOMFILTER => 'ROW', VERSIONS => '1', IN_MEMORY => 'false', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', COMPRESSION => 'NONE', MIN_VERS IONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} 3 row(s) in 0.1150 seconds hbase(main):004:0>
b、驗(yàn)證HA功能
1、hbase默認(rèn)的web管理端口是60010,兩個(gè)HMaster誰(shuí)先啟動(dòng)誰(shuí)就是主active節(jié)點(diǎn),10.10.40.212先啟動(dòng),10.10.20.64后啟動(dòng),web截圖如下:
2、停止10.10.40.212的HMaster進(jìn)程,查看10.10.20.64是否會(huì)提升為master
/etc/init.d/hbase-master stop
免責(zé)聲明:本站發(fā)布的內(nèi)容(圖片、視頻和文字)以原創(chuàng)、轉(zhuǎn)載和分享為主,文章觀點(diǎn)不代表本網(wǎng)站立場(chǎng),如果涉及侵權(quán)請(qǐng)聯(lián)系站長(zhǎng)郵箱:is@yisu.com進(jìn)行舉報(bào),并提供相關(guān)證據(jù),一經(jīng)查實(shí),將立刻刪除涉嫌侵權(quán)內(nèi)容。