您好,登錄后才能下訂單哦!
概述
logstash
logstash就是一根具備實(shí)時(shí)數(shù)據(jù)傳輸能力的管道,負(fù)責(zé)將數(shù)據(jù)信息從管道的輸入端傳輸?shù)焦艿赖妮敵龆?;與此同時(shí)這根管道還可以讓你根據(jù)自己的需求在中間加上濾網(wǎng),Logstash提供里很多功能強(qiáng)大的濾網(wǎng)以滿足你的各種應(yīng)用場(chǎng)景。
Nginx
Nginx(engine x) 是一個(gè)高性能的HTTP和反向代理服務(wù)器,也是一個(gè)IMAP/POP3/SMTP服務(wù)器。其將源代碼以類BSD許可證的形式發(fā)布,因它的穩(wěn)定性、豐富的功能集、示例配置文件和低系統(tǒng)資源的消耗而聞名。Nginx是一款輕量級(jí)的Web服務(wù)器/反向代理服務(wù)器及電子郵件(IMAP/POP3)代理服務(wù)器,并在一個(gè)BSD-like 協(xié)議下發(fā)行。其特點(diǎn)是占有內(nèi)存少,并發(fā)能力強(qiáng),事實(shí)上nginx的并發(fā)能力確實(shí)在同類型的網(wǎng)頁(yè)服務(wù)器中表現(xiàn)較好。
1.logstash安裝(jdk提前安裝1.8)
rpm -ivh logstash-6.6.2.rpm
2.上傳數(shù)據(jù)庫(kù)驅(qū)動(dòng)jar包
wget https://dev.mysql.com/get/Downloads/Connector-J/mysql-connector-java-5.1.48.tar.gz
mkdir -p /usr/share/logstash/vendor/jar/jdbc cd /usr/share/logstash/vendor/jar/jdbc [root@localhost soft]# cd /usr/share/logstash/vendor/jar/jdbc [root@localhost jdbc]# ll total 984 -rw-r--r--. 1 logstash logstash 1006959 Jul 11 19:43 mysql-connector-java-5.1.48-bin.jar [root@localhost jdbc]#
3.數(shù)據(jù)庫(kù)創(chuàng)建庫(kù) 并授權(quán)用戶
create database nginxlog; use nginxlog; CREATE TABLE `consumerlog` ( `client_ip` varchar(128) DEFAULT NULL, `log_time` timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP, `status` int(6) DEFAULT NULL, `http_referer` text, `AgentVersion` varchar(512) DEFAULT NULL, ) grant all on nginxlog.* to nginxlog@'%' identified by '123456';
4.安裝 logstash-output-jdbc插件
vim /usr/share/logstash/Gemfile # source "https://rubygems.org" 將國(guó)外的源注釋,換成國(guó)內(nèi)的 source "https://gems.ruby-china.com/"/usr/share/logstash/bin/logstash-plugin install logstash-output-jdbc
Validating logstash-output-jdbc Installing logstash-output-jdbc Installation successful /usr/share/logstash/bin/logstash-plugin list | grep jdbc logstash-input-jdbc logstash-output-jdbc
5.nginx日志格式設(shè)置
log_format access_log_json '{"client_ip":"$remote_addr","log_time":"$time_local","request":"$request","status":"$status","body_bytes_sent":"$body_bytes_sent","http_referer":"$http_referer","AgentVersion":"$http_user_agent","upstream_addr":"$upstream_addr","request_time":"$request_time","upstream_response_time":"$upstream_response_time"}';
6.
[root@localhost conf.d]# cat consumer_log.conf
/etc/logstash/conf.d
input{ file{ path => "/usr/local/tengine-2.1.2/logs/sxt-consumer.log" start_position => "beginning" stat_interval => "2" codec => "json" } } filter { if [status] != "200" { drop{} } } output{ jdbc{ connection_string => "jdbc:mysql://192.168.14.61/nginxlog?user=nginxlog&password=123456&useUnicode=true&characterEncoding=UTF8" statement => ["insert into consumerlog(client_ip,status,http_referer,AgentVersion) VALUES(?,?,?,?)", "client_ip","status","http_referer","AgentVersion"] } }
7.啟動(dòng)logstash
systemctl start logstash
8.驗(yàn)證數(shù)據(jù)
免責(zé)聲明:本站發(fā)布的內(nèi)容(圖片、視頻和文字)以原創(chuàng)、轉(zhuǎn)載和分享為主,文章觀點(diǎn)不代表本網(wǎng)站立場(chǎng),如果涉及侵權(quán)請(qǐng)聯(lián)系站長(zhǎng)郵箱:is@yisu.com進(jìn)行舉報(bào),并提供相關(guān)證據(jù),一經(jīng)查實(shí),將立刻刪除涉嫌侵權(quán)內(nèi)容。