溫馨提示×

溫馨提示×

您好，登錄后才能下訂單哦！

密碼登錄×
登錄注冊×
其他方式登錄
點(diǎn)擊 登錄注冊 即表示同意《億速云用戶服務(wù)條款》

Timeline Service v.2 (HDP3.1 )參數(shù)配置以及相關(guān)環(huán)境

發(fā)布時(shí)間：2020-07-08 07:41:43 來源：網(wǎng)絡(luò) 閱讀：1336 作者：Lynn_Yuan 欄目：大數(shù)據(jù)

Timeline Service v2 默認(rèn)集成嵌入HBase(HMaster、HRegionServer)，進(jìn)程啟動(dòng) User: yarn-ats

  1. 嵌入HBase配置文件路徑： 
    [yarn-ats@master1 embedded-yarn-ats-hbase]$ pwd
/usr/hdp/3.1.0.0-78/hadoop-yarn/conf/embedded-yarn-ats-hbase
[yarn-ats@master1 embedded-yarn-ats-hbase]$ ls -la
total 40
drwxr-xr-x 2 yarn-ats hadoop 4096 3月   9 10:32 .
drwxr-xr-x 4 root     root   4096 4月  23 19:32 ..
-rw-r--r-- 1 yarn-ats hadoop 1462 3月   7 09:22 hadoop-metrics2-hbase.properties
-rw-r--r-- 1 yarn-ats hadoop 3374 4月  23 20:41 hbase-env.sh
-rw-r--r-- 1 yarn-ats hadoop  818 3月   7 09:22 hbase_grant_permissions.sh
-rw-r--r-- 1 yarn-ats hadoop  411 3月   7 09:22 hbase-policy.xml
-rw-r--r-- 1 yarn-ats hadoop 5394 3月   7 09:22 hbase-site.xml
-rw-r--r-- 1 yarn-ats hadoop 5315 3月   7 09:22 log4j.properties

  2. hbase-site.xml文件

    <configuration  xmlns:xi="http://www.w3.org/2001/XInclude">

<property>
  <name>dfs.domain.socket.path</name>
  <value>/var/lib/hadoop-hdfs/dn_socket</value>
</property>

<property>
  <name>hbase.bucketcache.ioengine</name>
  <value></value>
</property>

<property>
  <name>hbase.bucketcache.percentage.in.combinedcache</name>
  <value></value>
</property>

<property>
  <name>hbase.bucketcache.size</name>
  <value></value>
</property>

<property>
  <name>hbase.client.keyvalue.maxsize</name>
  <value>1048576</value>
</property>

<property>
  <name>hbase.client.retries.number</name>
  <value>7</value>
</property>

<property>
  <name>hbase.client.scanner.caching</name>
  <value>100</value>
</property>

<property>
  <name>hbase.cluster.distributed</name>
  <value>true</value>
</property>

<property>
  <name>hbase.coprocessor.master.classes</name>
  <value></value>
</property>

<property>
  <name>hbase.coprocessor.region.classes</name>
  <value></value>
</property>

<property>
  <name>hbase.coprocessor.regionserver.classes</name>
  <value></value>
</property>

<property>
  <name>hbase.defaults.for.version.skip</name>
  <value>true</value>
</property>

<property>
  <name>hbase.hregion.majorcompaction</name>
  <value>604800000</value>
</property>

<property>
  <name>hbase.hregion.majorcompaction.jitter</name>
  <value>0.50</value>
</property>

<property>
  <name>hbase.hregion.max.filesize</name>
  <value>10737418240</value>
</property>

<property>
  <name>hbase.hregion.memstore.block.multiplier</name>
  <value>4</value>
</property>

<property>
  <name>hbase.hregion.memstore.flush.size</name>
  <value>134217728</value>
</property>

<property>
  <name>hbase.hregion.memstore.mslab.enabled</name>
  <value>true</value>
</property>

<property>
  <name>hbase.hstore.blockingStoreFiles</name>
  <value>10</value>
</property>

<property>
  <name>hbase.hstore.compaction.max</name>
  <value>10</value>
</property>

<property>
  <name>hbase.hstore.compactionThreshold</name>
  <value>3</value>
</property>

<property>
  <name>hbase.local.dir</name>
  <value>${hbase.tmp.dir}/local</value>
</property>

<property>
  <name>hbase.master.info.bindAddress</name>
  <value>0.0.0.0</value>
</property>

<property>
  <name>hbase.master.info.port</name>
  <value>17010</value>
</property>

<property>
  <name>hbase.master.namespace.init.timeout</name>
  <value>2400000</value>
</property>

<property>
  <name>hbase.master.port</name>
  <value>17000</value>
</property>

<property>
  <name>hbase.master.ui.readonly</name>
  <value>false</value>
</property>

<property>
  <name>hbase.master.wait.on.regionservers.timeout</name>
  <value>30000</value>
</property>

<property>
  <name>hbase.regionserver.executor.openregion.threads</name>
  <value>20</value>
</property>

<property>
  <name>hbase.regionserver.global.memstore.size</name>
  <value>0.4</value>
</property>

<property>
  <name>hbase.regionserver.handler.count</name>
  <value>30</value>
</property>

<property>
  <name>hbase.regionserver.info.port</name>
  <value>17030</value>
</property>

<property>
  <name>hbase.regionserver.port</name>
  <value>17020</value>
</property>

<property>
  <name>hbase.rootdir</name>
  <value>/atsv2/hbase/data</value>
</property>

<property>
  <name>hbase.rpc.protection</name>
  <value>authentication</value>
</property>

<property>
  <name>hbase.rpc.timeout</name>
  <value>90000</value>
</property>

<property>
  <name>hbase.security.authentication</name>
  <value>simple</value>
</property>

<property>
  <name>hbase.security.authorization</name>
  <value>false</value>
</property>

<property>
  <name>hbase.superuser</name>
  <value>yarn</value>
</property>

<property>
  <name>hbase.tmp.dir</name>
  <value>/tmp/hbase-${user.name}</value>
</property>

<property>
  <name>hbase.zookeeper.property.clientPort</name>
  <value>2181</value>
</property>

<property>
  <name>hbase.zookeeper.quorum</name>
  <value>manager.bigdata.com,master2.bigdata.com,master1.bigdata.com</value>
</property>

<property>
  <name>hbase.zookeeper.useMulti</name>
  <value>true</value>
</property>

<property>
  <name>hfile.block.cache.size</name>
  <value>0.4</value>
</property>

<property>
  <name>zookeeper.recovery.retry</name>
  <value>6</value>
</property>

<property>
  <name>zookeeper.session.timeout</name>
  <value>90000</value>
</property>

<property>
  <name>zookeeper.znode.parent</name>
  <value>/atsv2-hbase-unsecure</value>
</property>

</configuration>

  3. Zookeeper Cluster znode信息

    [zookeeper@master1 zookeeper-client]$ bin/zkCli.sh -server master1.bigdata.com:2181
[zk: master1.bigdata.com:2181(CONNECTED) 1] ls /atsv2-hbase-unsecure
[replication, meta-region-server, rs, splitWAL, backup-masters, table-lock, flush-table-proc, master-maintenance, online-snapshot, switch, master, running, draining, namespace, hbaseid, table]
[zk: master1.bigdata.com:2181(CONNECTED) 2] ls /atsv2-hbase-unsecure/rs
  4. UI Interface 
    http://master1.bigdata.com:17010/
  5. Table list 
    prod.timelineservice.app_flow
prod.timelineservice.application
prod.timelineservice.entity 
prod.timelineservice.flowactivity
prod.timelineservice.flowrun
prod.timelineservice.subapplication

    Timeline Service v.2 (HDP3.1 )參數(shù)配置以及相關(guān)環(huán)境
    TTL: 默認(rèn)30天

  6. Hbase shell執(zhí)行 
    hbase --config /usr/hdp/3.1.0.0-78/hadoop-yarn/conf/embedded-yarn-ats-hbase shell
hbase(main):004:0* status
1 active master, 0 backup masters, 1 servers, 0 dead, 124.0000 average load
Took 1.2255 seconds                                                                                                                                                    
hbase(main):005:0> list
TABLE                                                                                                                                                                  
prod.timelineservice.app_flow                                                                                                                                 
prod.timelineservice.application                                                                                                                                
prod.timelineservice.entity                                                                                                                                       
prod.timelineservice.flowactivity                                                                                                                               
prod.timelineservice.flowrun                                                                                                                                      
prod.timelineservice.subapplication                                                                                                                          
6 row(s)
Took 0.0862 seconds                                                                                                                                                    
=> ["prod.timelineservice.app_flow", "prod.timelineservice.application", "prod.timelineservice.entity", "prod.timelineservice.flowactivity", "prod.timelineservice.flowrun", "prod.timelineservice.subapplication"]
hbase(main):018:0> scan 'prod.timelineservice.app_flow', { COLUMNS => ['m'], LIMIT => 10 }
  7. HDFS存儲路徑
    /atsv2/hbase
向AI問一下細(xì)節(jié)

免責(zé)聲明：本站發(fā)布的內(nèi)容（圖片、視頻和文字）以原創(chuàng)、轉(zhuǎn)載和分享為主，文章觀點(diǎn)不代表本網(wǎng)站立場，如果涉及侵權(quán)請聯(lián)系站長郵箱：is@yisu.com進(jìn)行舉報(bào)，并提供相關(guān)證據(jù)，一經(jīng)查實(shí)，將立刻刪除涉嫌侵權(quán)內(nèi)容。

AI
助
手