溫馨提示×
您好，登錄后才能下訂單哦！
點擊 登錄注冊 即表示同意《億速云用戶服務(wù)條款》
您好，登錄后才能下訂單哦！
（一）測試的環(huán)境
agentd:192.168.180.22
ES：192.168.180.23
kibana：192.168.180.23
采用的拓撲：logstash -->ES-->kibana
（二）實施步驟:
（1）logstsh具體配置：
1，配置nginx日志格式，采用log_format格式:
log_format main '$remote_addr - $remote_user [$time_local] "$request" ' '$status $body_bytes_sent "$http_referer" ' '"$http_user_agent" "$http_x_forwarded_for"';
2,在logstash服務(wù)器下載IP地址歸類查詢庫
[root@localhost config]# cd /usr/local/logstash/config/ [root@localhost config]# wget http://geolite.maxmind.com/download/geoip/database/GeoLite2-City.mmdb. gz
3,配置logstash客戶端
[root@localhost config]# vim /usr/local/logstash/config/nginx-access.conf
input {
file {
path => "/opt/access.log"
type => "nginx"
start_position => "beginning"
}
}
filter {
grok {
match => { "message" => "%{IPORHOST:remote_addr} - - \[%{HTTPDATE:time_local}\] \"%{WO
RD:method} %{URIPATHPARAM:request} HTTP/%{NUMBER:httpversion}\" %{INT:status} %{INT:body_bytes_sent} %
{QS:http_referer} %{QS:http_user_agent}"
}
}
geoip {
source => "remote_addr"
target => "geoip"
database => "/usr/local/logstash/config/GeoLite2-City.mmdb"
add_field => ["[geoip][coordinates]","%{[geoip][longitude]}"]
add_field => ["[geoip][coordinates]","%{[geoip][latitude]}"]
}
}
output {
elasticsearch {
hosts => ["192.168.180.23:9200"]
manage_template => true
index => "logstash-map-%{+YYYY-MM}"
}
}備注：
geoip: IP查詢插件
source: 需要通過geoip插件處理的field，一般為ip，這里因為通過控制臺手動輸入的是ip所以直接填message，生成環(huán)境中如果查詢nginx訪問用戶，需先將客戶端ip過濾出來，然后這里填remote_addr即可
target: 解析后的Geoip地址數(shù)據(jù)，應(yīng)該存放在哪一個字段中，默認是geoip這個字段
database: 指定下載的數(shù)據(jù)庫文件
add_field: 這里兩行是添加經(jīng)緯度，地圖中地區(qū)顯示是根據(jù)經(jīng)緯度來識
。如果啟動正常的話，可以在kibana看到geoip相關(guān)的字段，如下圖:
3,啟動logstash客戶端并加載剛才的配置文件。
[root@localhost config]# /usr/local/logstash/bin/logstash -f nginx-access.conf
Sending Logstash's logs to /usr/local/logstash/logs which is now configured via log4j2.properties
[2017-06-20T22:55:23,801][INFO ][logstash.outputs.elasticsearch] Elasticsearch pool URLs updated {:changes=>{:removed=>[], :added=>[http://192.168.180.23:9200/]}}
[2017-06-20T22:55:23,805][INFO ][logstash.outputs.elasticsearch] Running health check to see if an Elasticsearch connection is working {:healthcheck_url=>http://192.168.180.23:9200/, :path=>"/"}
[2017-06-20T22:55:23,901][WARN ][logstash.outputs.elasticsearch] Restored connection to ES instance {:url=>#<URI::HTTP:0x4e54594d URL:http://192.168.180.23:9200/>}
[2017-06-20T22:55:23,909][INFO ][logstash.outputs.elasticsearch] Using mapping template from {:path=>nil}
[2017-06-20T22:55:23,947][INFO ][logstash.outputs.elasticsearch] Attempting to install template {:manage_template=>{"template"=>"logstash-*", "version"=>50001, "settings"=>{"index.refresh_interval"=>"5s"}, "mappings"=>{"_default_"=>{"_all"=>{"enabled"=>true, "norms"=>false}, "dynamic_templates"=>[{"message_field"=>{"path_match"=>"message", "match_mapping_type"=>"string", "mapping"=>{"type"=>"text", "norms"=>false}}}, {"string_fields"=>{"match"=>"*", "match_mapping_type"=>"string", "mapping"=>{"type"=>"text", "norms"=>false, "fields"=>{"keyword"=>{"type"=>"keyword"}}}}}], "properties"=>{"@timestamp"=>{"type"=>"date", "include_in_all"=>false}, "@version"=>{"type"=>"keyword", "include_in_all"=>false}, "geoip"=>{"dynamic"=>true, "properties"=>{"ip"=>{"type"=>"ip"}, "location"=>{"type"=>"geo_point"}, "latitude"=>{"type"=>"half_float"}, "longitude"=>{"type"=>"half_float"}}}}}}}}
[2017-06-20T22:55:23,955][INFO ][logstash.outputs.elasticsearch] New Elasticsearch output {:class=>"LogStash::Outputs::ElasticSearch", :hosts=>[#<URI::Generic:0x1e098115 URL://192.168.180.23:9200>]}
[2017-06-20T22:55:24,065][INFO ][logstash.filters.geoip ] Using geoip database {:path=>"/usr/local/logstash/config/GeoLite2-City.mmdb"}
[2017-06-20T22:55:24,094][INFO ][logstash.pipeline ] Starting pipeline {"id"=>"main", "pipeline.workers"=>4, "pipeline.batch.size"=>125, "pipeline.batch.delay"=>5, "pipeline.max_inflight"=>500}
[2017-06-20T22:55:24,275][INFO ][logstash.pipeline ] Pipeline main started
[2017-06-20T22:55:24,369][INFO ][logstash.agent ] Successfully started Logstash API endpoint {:port=>9600}（2）Kibana配置.
1,編輯修改kibana的配置文件kibana.yml在最后添加如下：
# The default locale. This locale can be used in certain circumstances to substitute any missing
# translations.
#i18n.defaultLocale: "en"
tilemap.url: 'http://webrd02.is.autonavi.com/appmaptile?lang=zh_cn&size=1&scale=1&style=7&x={x}&y={y}&
z={z}'2,重啟kibana服務(wù)。
[root@localhost bin]# /usr/local/kibana/bin/kibana & [1] 10631 [root@localhost bin]# ps -ef|grep kibana root 10631 7795 21 10:52 pts/0 00:00:02 /usr/local/kibana/bin/../node/bin/node --no-warnings /usr/local/kibana/bin/../src/cli root 10643 7795 0 10:52 pts/0 00:00:00 grep --color=auto kibana [root@localhost bin]# log [02:52:59.297] [info][status][plugin:kibana@5.4.0] Status changed from uninitialized to green - Ready log [02:52:59.445] [info][status][plugin:elasticsearch@5.4.0] Status changed from uninitialized to yellow - Waiting for Elasticsearch log [02:52:59.482] [info][status][plugin:console@5.4.0] Status changed from uninitialized to green - Ready log [02:52:59.512] [info][status][plugin:elasticsearch@5.4.0] Status changed from yellow to green - Kibana index ready log [02:52:59.513] [info][status][plugin:metrics@5.4.0] Status changed from uninitialized to green - Ready log [02:53:00.075] [info][status][plugin:timelion@5.4.0] Status changed from uninitialized to green - Ready log [02:53:00.080] [info][listening] Server running at http://192.168.180.23:5601 log [02:53:00.081] [info][status][ui settings] Status changed from uninitialized to green - Ready
3，創(chuàng)建nginx的訪問索引lostash-map*。具體步驟如下：ip:5601--->Management--Index Patterns--->+--->在Index name or pattern中添加logstash-map*--->create。具體如下圖：
4,創(chuàng)建Visualize。具體步驟如下：Visalize--->+--->Maps(Tile Maps)
已上就是簡單的步驟。
免責聲明：本站發(fā)布的內(nèi)容（圖片、視頻和文字）以原創(chuàng)、轉(zhuǎn)載和分享為主，文章觀點不代表本網(wǎng)站立場，如果涉及侵權(quán)請聯(lián)系站長郵箱：is@yisu.com進行舉報，并提供相關(guān)證據(jù)，一經(jīng)查實，將立刻刪除涉嫌侵權(quán)內(nèi)容。
億速云公眾號
手機網(wǎng)站二維碼