kafka配置

1.三節(jié)點(diǎn)主機(jī)環(huán)境：
#Red Hat Enterprise Linux Server release 7.5
192.168.154.128   my1.db.com   my1
192.168.154.129   my2.db.com   my2
192.168.154.130   my3.db.com   my3

2.配置jdk：
tar xvzf jdk-8u201-linux-x64.tar.gz
mv jdk1.8.0_201 /usr/local/jdk
vi .bash_profile
JAVA_HOME=/usr/local/jdk
PATH=$PATH:$HOME/bin:$JAVA_HOME/bin
source .bash_profile

3.下載zookeeper：
http://zookeeper.apache.org/releases.html
tar xvzf zookeeper-3.4.13.tar.gz
mv zookeeper-3.4.13 /usr/local/zookeeper

4.配置zookeeper：

編輯配置文件：
vi /usr/local/zookeeper/conf/zoo.cfg
tickTime=2000
dataDir=/usr/local/zookeeper/data
clientPort=2181
initLimit=5
syncLimit=2
dataLogDir=/usr/local/zookeeper/log
server.1=my1:2888:3888
server.2=my2:2888:3888
server.3=my3:2888:3888

創(chuàng)建目錄：
mkdir -p /usr/local/zookeeper/data/
mkdir -p /usr/local/zookeeper/log/

echo 1 > /usr/local/zookeeper/data/myid
echo 2 > /usr/local/zookeeper/data/myid
echo 3 > /usr/local/zookeeper/data/myid

5.啟動(dòng)zookeeper：
/usr/local/zookeeper/bin/zkServer.sh start /usr/local/zookeeper/conf/zoo.cfg

查看狀態(tài)：
/usr/local/zookeeper/bin/zkServer.sh status /usr/local/zookeeper/conf/zoo.cfg


6.下載kafka：
http://kafka.apache.org/downloads
tar xvzf kafka_2.12-2.0.0.tgz
mv kafka_2.12-2.0.0 /usr/local/kafka

mkdir -p /usr/local/kafka/data

編輯配置文件：
vi /usr/local/kafka/config/server.properties
broker.id=1
listeners=PLAINTEXT://my1:9092
zookeeper.connect=192.168.154.128:2181,192.168.154.129:2181,192.168.154.130:2181
delete.topic.enable=true

啟動(dòng)kafka：
/usr/local/kafka/bin/kafka-server-start.sh -daemon /usr/local/kafka/config/server.properties

查看確認(rèn)：
jps

7.測(cè)試kafka：
創(chuàng)建一個(gè)測(cè)試topic：
/usr/local/kafka/bin/kafka-topics.sh --zookeeper 192.168.154.128:2181,192.168.154.129:2181,192.168.154.130:2181 --create --topic test-topic --partitions 3 --replication-factor 3
查看topic：
/usr/local/kafka/bin/kafka-topics.sh --zookeeper 192.168.154.128:2181,192.168.154.129:2181,192.168.154.130:2181 --list
/usr/local/kafka/bin/kafka-topics.sh --zookeeper 192.168.154.128:2181,192.168.154.129:2181,192.168.154.130:2181 --describe --topic test-topic

刪除topic：
/usr/local/kafka/bin/kafka-topics.sh --zookeeper 192.168.154.128:2181,192.168.154.129:2181,192.168.154.130:2181 --delete --topic test-topic

producer發(fā)送消息：
/usr/local/kafka/bin/kafka-console-producer.sh --broker-list my1:9092,my2:9092,my3:9092 --topic test-topic

consumer消費(fèi)消息：
/usr/local/kafka/bin/kafka-console-consumer.sh --bootstrap-server my1:9092,my2:9092,my3:9092 --topic test-topic --from-beginning

生產(chǎn)者吞吐量測(cè)試：
/usr/local/kafka/bin/kafka-producer-perf-test.sh --topic test-topic --num-records 500000 --record-size 200 --throughput -1 --producer-props bootstrap.servers=192.168.154.128:9092,192.168.154.129:9092,192.168.154.130:9092 acks=1