您好,登錄后才能下訂單哦!
大數(shù)據(jù)定義是:大量的非結(jié)構(gòu)話的數(shù)據(jù)。量要大,要非結(jié)構(gòu)化。
Hadoop 分三部分組成,1.hdfs hadoop分布式文件系統(tǒng)。2.MapReduce 分布式計算。3.hive 分布式存儲。
操作系統(tǒng):centos6.5 64
環(huán)境搭建:1.安裝Hadoop,2.安裝mysql, 3.安裝hive,4,安裝jdk
Hadoop安裝:
下載Hadoop1.0.4,hadoop-1.0.4.tar.gz。
wget http://archive.apache.org/dist/hadoop/core/hadoop-1.0.4/hadoop-1.0.4.tar.gz
解壓:
tar xzvf hadoop-1.0.4.tar.gz -C /usr
修改3個配置文件
cd /usr/hadoop-1.0.4/conf/
vi core-site.xml
<?xml version="1.0"?>
<?xml-stylesheet type="text/xsl" href="configuration.xsl"?>
<!-- Put site-specific property overrides in this file. -->
<configuration>
<property>
<name>fs.default.name</name>
<value>hdfs://localhost:9000</value>
</property>
</configuration>
vi mapred-site.xml
<?xml version="1.0"?>
<?xml-stylesheet type="text/xsl" href="configuration.xsl"?>
<!-- Put site-specific property overrides in this file. -->
<configuration>
<property>
<name>mapred.job.tracker</name>
<value>localhost:9001</value>
</property>
</configuration>
vi hdfs-site.xml
<?xml version="1.0"?>
<?xml-stylesheet type="text/xsl" href="configuration.xsl"?>
<!-- Put site-specific property overrides in this file. -->
<configuration>
<property>
<name>dfs.name.dir</name>
<value>/home/longlong/temp/log1,/home/longlong/temp/log2</value>
</property>
<property>
<name>dfs.data.dir</name>
<value>/home/longlong/temp/data1,/home/longlong/temp/data2</value>
</property>
<property>
<name>dfs.replication</name>
<value>2</value>
</property>
</configuration>
配置環(huán)境變量,此處粘貼了java,hive,hadoop的環(huán)境變量
vi /etc/profile
JAVA_HOME=/home/Hadoop/jdk1.6.0_45
CLASSPATH=$JAVA_HOME/jre/lib/rt.jar
HADOOP_HOME=/usr/hadoop-1.0.4
HIVE_HOME=/usr/hive
PATH=$HADOOP_HOME/bin:$PATH:$JAVA_HOME/bin:$HIVE_HOME/bin
安裝jdk:
下載jdk
wget http://download.oracle.com/otn-pub/java/jdk/6u45-b06/jdk-6u45-linux-x64.bin
chmod +x jdk-6u45-linux-x64.bin
./jdk-6u45-linux-x64.bin
配置 vi hadoop-env.sh,末尾添加jdkhome
export JAVA_HOME=/home/Hadoop/jdk1.6.0_45
進(jìn)入bin目錄配置 vi hadoop-config.sh
export HADOOP_HOME=${HADOOP_PREFIX}
export HADOOP_HOME_WARN_SUPPRESS=1
格式化
./hadoop namenode -format
2.mysql 安裝,
3.hive安裝,
下載
wget http://mirrors.cnnic.cn/apache/hive/hive-0.13.1/apache-hive-0.13.1-bin.tar.gz
tar -xf apache-hive-0.13.1-bin.tar.gz
mv apache-hive-0.13.1-bin hive
更換元數(shù)據(jù)庫
cd conf/
touch hive-site.xml
vi hive-site.xml
<?xml version="1.0" ?>
<?xml-stylesheet type="text/xsl" href="configuration.xsl"?>
<configuration>
<property>
<name>javax.jdo.option.ConnectionURL</name>
<value>jdbc:mysql://localhost:3306/hive</value>
</property>
<property>
<name>javax.jdo.option.ConnectionDriverName</name>
<value>com.mysql.jdbc.Driver</value>
</property>
<property>
<name>javax.jdo.option.ConnectionUserName</name>
<value>root</value>
</property>
<property>
<name>javax.jdo.option.ConnectionPassword</name>
<value>root</value>
</property>
</configuration>
下載mysql驅(qū)動,
wget http://cdn.mysql.com/Downloads/Connector-J/mysql-connector-java-5.0.8.tar.gz
tar -xzvf mysql-connector-java-5.0.8.tar.gz
cd mysql-connector-java-5.0.8
cp mysql-connector-java-5.0.8-bin.jar /usr/hive/lib/
啟動
1.hdfs 啟動
cd /usr/hadoop-1.0.4/bin
./start-all.sh
password/操作系統(tǒng)密碼
2.mysql 啟動
service mysqld start
mysql -uroot -proot
3.hive啟動
cd /usr/hive/bin
./hive
免責(zé)聲明:本站發(fā)布的內(nèi)容(圖片、視頻和文字)以原創(chuàng)、轉(zhuǎn)載和分享為主,文章觀點(diǎn)不代表本網(wǎng)站立場,如果涉及侵權(quán)請聯(lián)系站長郵箱:is@yisu.com進(jìn)行舉報,并提供相關(guān)證據(jù),一經(jīng)查實(shí),將立刻刪除涉嫌侵權(quán)內(nèi)容。