溫馨提示×

溫馨提示×

您好,登錄后才能下訂單哦!

密碼登錄×
登錄注冊×
其他方式登錄
點(diǎn)擊 登錄注冊 即表示同意《億速云用戶服務(wù)條款》

安裝hadoop

發(fā)布時間:2020-07-08 06:45:29 來源:網(wǎng)絡(luò) 閱讀:532 作者:liufu1103 欄目:大數(shù)據(jù)

1:安裝包

http://hadoop.apache.org/releases.html

http://www.oracle.com/technetwork/java/javase/downloads/jdk8-downloads-2133151.html

hadoop-2.6.4 (binary)

jdk-8u102-linux-x64.tar.gz


2:節(jié)點(diǎn)信息

centos7 * 3

master 10.10.0.115

slave1 10.10.0.116

slave2 10.10.0.117


3:安裝過程

3.1 節(jié)下初始化(略)

     主機(jī)名  防火墻  selinux  等

3.2 節(jié)點(diǎn)互信

[root@master ~]#ssh-keygen

[root@master ~]#cat /root/.ssh/id_rsa.pub >  /root/.ssh/authorized_keys 

[root@master ~]#scp -r /root/.ssh slave1:/root/

[root@master ~]#scp -r /root/.ssh slave2:/root/


3.3 在所有節(jié)點(diǎn)安裝jdk

[root@master ~]#tar -zxvf jdk-8u102-linux-x64.tar.gz

[root@master ~]#mkdir   /usr/soft

[root@master ~]#mv jdk1.8.0_102 /usr/soft/

[root@master ~]#vim /etc/profile

   ...

   export JAVA_HOME=/usr/soft/jdk1.8.0_102
   export CLASSPATH=.:$JAVA_HOME/jre/lib/rt.jar:$JAVA_HOME/lib/dt.jar:$JAVA_HOME/lib/tools.jar
   export PATH=$PATH:$JAVA_HOME/bin


[root@master ~]#source /etc/profile

[root@master ~]#java -version
java version "1.8.0_102"
Java(TM) SE Runtime Environment (build 1.8.0_102-b14)
Java HotSpot(TM) 64-Bit Server VM (build 25.102-b14, mixed mode)


其它節(jié)點(diǎn)同樣操作


3.4 安裝hadoop

先在master上安裝配置,然后scp到其它節(jié)點(diǎn)即可

[root@master ~]#tar -zxvf hadoop-2.6.4.tar.gz

[root@master ~]#mv hadoop-2.6.4 /usr/soft/

[root@master ~]#cd /usr/soft/hadoop-2.6.4/

[root@master ~]#mkdir hdfs

[root@master ~]#mkdir hdfs/data

[root@master ~]#mkdir hdfs/name

[root@master ~]#mkdir tmp

[root@master ~]# cat /usr/soft/hadoop-2.6.4/etc/hadoop/core-site.xml

<configuration>
   <property>
        <name>fs.defaultFS</name>
        <value>hdfs://10.10.0.115:9000</value>
    </property>
    <property>
        <name>hadoop.tmp.dir</name>
        <value>file:/usr/soft/hadoop-2.6.4/tmp</value>
    </property>
    <property>
        <name>io.file.buffer.size</name>
        <value>131702</value>
    </property>
</configuration>


[root@master ~]# cat /usr/soft/hadoop-2.6.4/etc/hadoop/hdfs-site.xml

<configuration>
    <property>
        <name>dfs.namenode.name.dir</name>
        <value>file:/usr/soft/hadoop-2.6.4/hdfs/name</value>
    </property>
    <property>
        <name>dfs.datanode.data.dir</name>
        <value>file:/usr/soft/hadoop-2.6.4/hdfs/data</value>
    </property>
    <property>
        <name>dfs.replication</name>
        <value>2</value>
    </property>
    <property>
        <name>dfs.namenode.secondary.http-address</name>
        <value>10.10.0.115:9001</value>
    </property>
    <property>
    <name>dfs.webhdfs.enabled</name>
    <value>true</value>
    </property>
</configuration>


 [root@master ~]#cp /usr/soft/hadoop-2.6.4/etc/hadoop/mapred-site.xml.template   /usr/soft/hadoop-2.6.4/etc/hadoop/mapred-site.xml


[root@master ~]# cat /usr/soft/hadoop-2.6.4/etc/hadoop/mapred-site.xml

<configuration>
   <property>
        <name>mapreduce.framework.name</name>
        <value>yarn</value>
    </property>
    <property>
        <name>mapreduce.jobhistory.address</name>
        <value>10.10.0.115:10020</value>
    </property>
    <property>
        <name>mapreduce.jobhistory.webapp.address</name>
        <value>10.10.0.115:19888</value>
    </property>
    <property>
        <name>yarn.nodemanager.aux-services</name>
        <value>mapreduce_shuffle</value>
    </property>
    <property>
        <name>yarn.nodemanager.auxservices.mapreduce.shuffle.class</name>
        <value>org.apache.hadoop.mapred.ShuffleHandler</value>
    </property>
    <property>
        <name>yarn.resourcemanager.address</name>
        <value>10.10.0.115:8032</value>
    </property>
    <property>
        <name>yarn.resourcemanager.scheduler.address</name>
        <value>10.10.0.115:8030</value>
    </property>
    <property>
        <name>yarn.resourcemanager.resource-tracker.address</name>
        <value>10.10.0.115:8031</value>
    </property>
    <property>
        <name>yarn.resourcemanager.admin.address</name>
        <value>10.10.0.115:8033</value>
    </property>
    <property>
        <name>yarn.resourcemanager.webapp.address</name>
        <value>10.10.0.115:8088</value>
    </property>
    <property>
        <name>yarn.nodemanager.resource.memory-mb</name>
        <value>768</value>
    </property>
</configuration>


[root@master ~]# cat /usr/soft/hadoop-2.6.4/etc/hadoop/hadoop-env.sh

...

export JAVA_HOME=/usr/soft/jdk1.8.0_102


[root@master ~]# cat /usr/soft/hadoop-2.6.4/etc/hadoop/yarn-env.sh

...

export JAVA_HOME=/usr/soft/jdk1.8.0_102


[root@master ~]# cat  /usr/soft/hadoop-2.6.4/etc/hadoop/slaves
10.10.0.116
10.10.0.117


[root@master ~]#scp -r  /usr/soft/jdk1.8.0_102 slave1:/usr/soft/

[root@master ~]#scp -r  /usr/soft/jdk1.8.0_102 slave2:/usr/soft/


[root@master ~]#/usr/soft/hadoop-2.6.4/bin/hdfs namenode -format

[root@master ~]# /usr/soft/hadoop-2.6.4/sbin/start-all.sh



瀏覽器打開http://10.10.0.115:8088/
瀏覽器打開http://10.10.0.115:50070/








  





向AI問一下細(xì)節(jié)

免責(zé)聲明:本站發(fā)布的內(nèi)容(圖片、視頻和文字)以原創(chuàng)、轉(zhuǎn)載和分享為主,文章觀點(diǎn)不代表本網(wǎng)站立場,如果涉及侵權(quán)請聯(lián)系站長郵箱:is@yisu.com進(jìn)行舉報,并提供相關(guān)證據(jù),一經(jīng)查實(shí),將立刻刪除涉嫌侵權(quán)內(nèi)容。

AI