hadoop從集群中怎么移除一個(gè)節(jié)點(diǎn)

本篇內(nèi)容主要講解“hadoop從集群中怎么移除一個(gè)節(jié)點(diǎn)”，感興趣的朋友不妨來看看。本文介紹的方法操作簡單快捷，實(shí)用性強(qiáng)。下面就讓小編來帶大家學(xué)習(xí)“hadoop從集群中怎么移除一個(gè)節(jié)點(diǎn)”吧!

1.首先檢查整個(gè)集群的Average block replication，如果大于2，那即使直接拔出一個(gè)節(jié)點(diǎn)也不會丟失數(shù)據(jù)

hadoop fsck / 檢查集群的文件系統(tǒng)狀況

可以手動設(shè)置文件的冗余倍數(shù)，為了安全備份，可對關(guān)鍵數(shù)據(jù)hadoop fs -setrep -w 3 -R <path>

2.給下面兩個(gè)文件增加配置，在excludes中填寫要解任的節(jié)點(diǎn)名

mapred-site.xml

<property>

  <name>mapred.hosts</name>

  <value></value>

  <description>Names a file that contains the list of nodes that may

  connect to the jobtracker.  If the value is empty, all hosts are

  permitted.</description>

</property>

<property>

  <name>mapred.hosts.exclude</name>

  <value>HADOOP_HOME/conf/excludes</value>

  <description>Names a file that contains the list of hosts that

  should be excluded by the jobtracker.  If the value is empty, no

  hosts are excluded.</description>

</property>

hdfs-site.xml

<property>

  <name>dfs.hosts</name>

  <value></value>

  <description>Names a file that contains a list of hosts that are

  permitted to connect to the namenode. The full pathname of the file

  must be specified.  If the value is empty, all hosts are

  permitted.</description>

</property>

<property>

  <name>dfs.hosts.exclude</name>

  <value>HADOOP_HOME/conf/excludes</value>

  <description>Names a file that contains a list of hosts that are

  not permitted to connect to the namenode.  The full pathname of the

  file must be specified.  If the value is empty, no hosts are

  excluded.</description>

</property> 

excludes 文件里面配置機(jī)器的hostname即可。

run on namenode:   hadoop dfsadmin -refreshNodes  

run on jobtracker:   hadoop mradmin -refreshNodes 

“hadoop dfsadmin -refreshNodes”會觸發(fā)Decommission過程，在Decommission過程，集群會將Decommission節(jié)點(diǎn)上的數(shù)據(jù)冗余到其他幾點(diǎn)上，

到此，相信大家對“hadoop從集群中怎么移除一個(gè)節(jié)點(diǎn)”有了更深的了解，不妨來實(shí)際操作一番吧！這里是億速云網(wǎng)站，更多相關(guān)內(nèi)容可以進(jìn)入相關(guān)頻道進(jìn)行查詢，關(guān)注我們，繼續(xù)學(xué)習(xí)！