win7搭建為偽分布式hadoop的步驟

這篇文章主要介紹“win7搭建為偽分布式hadoop的步驟”，在日常操作中，相信很多人在win7搭建為偽分布式hadoop的步驟問題上存在疑惑，小編查閱了各式資料，整理出簡單好用的操作方法，希望對大家解答”win7搭建為偽分布式hadoop的步驟”的疑惑有所幫助！接下來，請跟著小編一起來學習吧！

window7下安裝hadoop [32位]
1.下載hadoop
hadoop-2.2.0.tar.gz
解壓
2.配置環(huán)境變量hadoop
3.修改%HADOOP_HOME%\etc\hadoop的hadoop-env.sh
export JAVA_HOME=D:\Java\jdk1.6.0_10#不能有空格
末尾添加[會出錯就不添加]
set HADOOP_PREFIX=E:\hadoop\hadoop-2.2.0\
set HADOOP_CONF_DIR=%HADOOP_PREFIX%\etc\hadoop  
set YARN_CONF_DIR=%HADOOP_CONF_DIR%  
set PATH=%PATH%;%HADOOP_PREFIX%\bin 

下載hadoop-common-2.2.0-bin-master

把里面的文件拷貝到hadoop/bin下

4.配置core-site.xml
<?xml version="1.0"?>
<?xml-stylesheet type="text/xsl" href="configuration.xsl"?>
<!-- Put site-specific property overrides in this file. -->
<configuration>  
      <property>  
        <name>fs.default.name</name>  
        <value>hdfs://0.0.0.0:19000</value>  
      </property>  
 </configuration>  
 
5.配置hdfs-site.xml

  <?xml version="1.0"?>
<?xml-stylesheet type="text/xsl" href="configuration.xsl"?>
<!-- Put site-specific property overrides in this file. -->

    <configuration>  
      <property>  
        <name>dfs.replication</name>  
        <value>1</value>  
      </property>  
    </configuration>  

    
6.配置mapred-site.xml

<?xml version="1.0"?>
<?xml-stylesheet type="text/xsl" href="configuration.xsl"?>

<!-- Put site-specific property overrides in this file. -->

    <configuration>  
      
       <property>  
         <name>mapreduce.job.user.name</name>  
         <value>%USERNAME%</value>  
       </property>  
      
       <property>  
         <name>mapreduce.framework.name</name>  
         <value>yarn</value>  
       </property>  
      
      <property>  
        <name>yarn.apps.stagingDir</name>  
        <value>/user/%USERNAME%/staging</value>  
      </property>  
      
      <property>  
        <name>mapreduce.jobtracker.address</name>  
        <value>local</value>  
      </property>  
      
    </configuration>  


7.創(chuàng)建yarn-site.xml文件
<?xml version="1.0"?>
<!--
  Licensed under the Apache License, Version 2.0 (the "License");
  you may not use this file except in compliance with the License.
  You may obtain a copy of the License at

    http://www.apache.org/licenses/LICENSE-2.0

  Unless required by applicable law or agreed to in writing, software
  distributed under the License is distributed on an "AS IS" BASIS,
  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
  See the License for the specific language governing permissions and
  limitations under the License. See accompanying LICENSE file.
-->

    <configuration>  
      <property>  
        <name>yarn.server.resourcemanager.address</name>  
        <value>0.0.0.0:8020</value>  
      </property>  
      
      <property>  
        <name>yarn.server.resourcemanager.application.expiry.interval</name>  
        <value>60000</value>  
      </property>  
      
      <property>  
        <name>yarn.server.nodemanager.address</name>  
        <value>0.0.0.0:45454</value>  
      </property>  
      
      <property>  
        <name>yarn.nodemanager.aux-services</name>  
        <value>mapreduce_shuffle</value>  
      </property>  
      
      <property>  
        <name>yarn.nodemanager.aux-services.mapreduce.shuffle.class</name>  
        <value>org.apache.hadoop.mapred.ShuffleHandler</value>  
      </property>  
      
      <property>  
        <name>yarn.server.nodemanager.remote-app-log-dir</name>  
        <value>/app-logs</value>  
      </property>  
      
      <property>  
        <name>yarn.nodemanager.log-dirs</name>  
        <value>/dep/logs/userlogs</value>  
      </property>  
      
      <property>  
        <name>yarn.server.mapreduce-appmanager.attempt-listener.bindAddress</name>  
        <value>0.0.0.0</value>  
      </property>  
      
      <property>  
        <name>yarn.server.mapreduce-appmanager.client-service.bindAddress</name>  
        <value>0.0.0.0</value>  
      </property>  
      
      <property>  
        <name>yarn.log-aggregation-enable</name>  
        <value>true</value>  
      </property>  
      
      <property>  
        <name>yarn.log-aggregation.retain-seconds</name>  
        <value>-1</value>  
      </property>  
      
      <property>  
        <name>yarn.application.classpath</name>  
        <value>%HADOOP_CONF_DIR%,%HADOOP_COMMON_HOME%/share/hadoop/common/*,%HADOOP_COMMON_HOME%/share/hadoop/common/lib/*,%HADOOP_HDFS_HOME%/share/hadoop/hdfs/*,%HADOOP_HDFS_HOME%/share/hadoop/hdfs/lib/*,%HADOOP_MAPRED_HOME%/share/hadoop/mapreduce/*,%HADOOP_MAPRED_HOME%/share/hadoop/mapreduce/lib/*,%HADOOP_YARN_HOME%/share/hadoop/yarn/*,%HADOOP_YARN_HOME%/share/hadoop/yarn/lib/*</value>  
      </property>  
    </configuration>  

    切換到E:\hadoop\hadoop-2.2.0\etc\hadoop運行hadoop-env.cmd腳本，設置當前命令窗口執(zhí)行環(huán)境變量
   執(zhí)行 hdfs namenode -format
   16/02/27 11:36:23 INFO util.GSet: VM type       = 32-bit
16/02/27 11:36:23 INFO util.GSet: 0.029999999329447746% max memory = 992.3 MB
16/02/27 11:36:23 INFO util.GSet: capacity      = 2^16 = 65536 entries
16/02/27 11:36:24 INFO common.Storage: Storage directory \tmp\hadoop-goudcheng\d
fs\name has been successfully formatted.
16/02/27 11:36:24 INFO namenode.FSImage: Saving image file \tmp\hadoop-goudcheng
\dfs\name\current\fsimage.ckpt_0000000000000000000 using no compression
16/02/27 11:36:24 INFO namenode.FSImage: Image file \tmp\hadoop-goudcheng\dfs\na
me\current\fsimage.ckpt_0000000000000000000 of size 201 bytes saved in 0 seconds
.
16/02/27 11:36:24 INFO namenode.NNStorageRetentionManager: Going to retain 1 ima
ges with txid >= 0
16/02/27 11:36:24 INFO util.ExitUtil: Exiting with status 0
16/02/27 11:36:24 INFO namenode.NameNode: SHUTDOWN_MSG:
/************************************************************
SHUTDOWN_MSG: Shutting down NameNode at CDCH20100020-5/172.31.168.244

到此，關于“win7搭建為偽分布式hadoop的步驟”的學習就結束了，希望能夠解決大家的疑惑。理論與實踐的搭配能更好的幫助大家學習，快去試試吧！若想繼續(xù)學習更多相關知識，請繼續(xù)關注億速云網(wǎng)站，小編會繼續(xù)努力為大家?guī)砀鄬嵱玫奈恼拢?/p>