您好,登錄后才能下訂單哦!
分布式安裝(至少三臺主機):
環(huán)境所需軟件:
CentOS7
hadoop-2.7.3.tar.gz
jdk-8u102-linux-x64.tar.gz
安裝前準(zhǔn)備工作:
配置免密鑰登陸
cd
ssh-keygen -t rsa
一直回車,直到結(jié)束
ssh-copy-id .ssh/id_rsa.pub bigdata1
ssh-copy-id .ssh/id_rsa.pub bigdata2
ssh-copy-id .ssh/id_rsa.pub bigdata3
同步時間
通過設(shè)置計劃任務(wù)實現(xiàn)各主機間的時間同步
vim /etc/crontab
0 0 1 root ntpdate -s time.windows.com
或者部署一個時間服務(wù)器實現(xiàn)同步,這里就不詳細講解了
(*)hdfs-site.xml
<!--數(shù)據(jù)塊的冗余度,默認(rèn)是3-->
<property>
<name>dfs.replication</name>
<value>2</value>
</property>
<!--是否開啟HDFS的權(quán)限檢查,默認(rèn):true-->
<!--
<property>
<name>dfs.permissions</name>
<value>false</value>
</property>
-->
core-site.xml
<!--NameNode的地址-->
<property>
<name>fs.defaultFS</name>
<value>hdfs://bigdata1:9000</value>
</property>
<!--HDFS數(shù)據(jù)保存的目錄,默認(rèn)是Linux的tmp目錄-->
<property>
<name>hadoop.tmp.dir</name>
<value>/root/training/hadoop-2.7.3/tmp</value>
</property>
mapred-site.xml
<!--MR程序運行的容器是Yarn-->
<property>
<name>mapreduce.framework.name</name>
<value>yarn</value>
</property>
yarn-site.xml
<!--ResourceManager的地址-->
<property>
<name>yarn.resourcemanager.hostname</name>
<value>bigdata1</value>
</property>
<!--NodeManager運行MR任務(wù)的方式-->
<property>
<name>yarn.nodemanager.aux-services</name>
<value>mapreduce_shuffle</value>
</property>
對NameNode進行格式化: hdfs namenode -format
日志:Storage directory /root/training/hadoop-2.7.3/tmp/dfs/name has been successfully formatted.
scp -r /root/training/hadoop-2.7.3 bigdata2:/root/training/hadoop-2.7.3
scp -r /root/training/hadoop-2.7.3 bigdata3:/root/training/hadoop-2.7.3
啟動:start-all.sh = start-dfs.sh + start-yarn.sh
驗證
(*)命令行:hdfs dfsadmin -report
(*)網(wǎng)頁:HDFS:http://192.168.157.12:50070/
Yarn:http://192.168.157.12:8088
(*)Demo:測試MapReduce程序
example: /root/training/hadoop-2.7.3/share/hadoop/mapreduce/hadoop-mapreduce-examples-2.7.3.jar
hadoop jar hadoop-mapreduce-examples-2.7.3.jar wordcount /input/data.txt /output/wc1204
免責(zé)聲明:本站發(fā)布的內(nèi)容(圖片、視頻和文字)以原創(chuàng)、轉(zhuǎn)載和分享為主,文章觀點不代表本網(wǎng)站立場,如果涉及侵權(quán)請聯(lián)系站長郵箱:is@yisu.com進行舉報,并提供相關(guān)證據(jù),一經(jīng)查實,將立刻刪除涉嫌侵權(quán)內(nèi)容。