Docker-compose部署ELK

? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?Docker-compose部署單機ELK

環(huán)境

主機IP 192.168.0.9

Docker version 19.03.2

docker-compose version 1.24.0-rc1

elasticsearch version 6.6.1

kibana version 6.6.1

logstash version 6.6.1

一、ELK-dockerfile文件編寫及配置文件

● elasticsearch

1、elasticsearch-dockerfile

FROM?centos:latest
ADD?elasticsearch-6.6.1.tar.gz??/usr/local/
COPY?elasticsearch.yml?/usr/local/elasticsearch-6.6.1/config/
COPY?jdk1.8?/usr/local/
ENV?JAVA_HOME=/usr/local/jdk1.8
ENV?CLASSPATH=$CLASSPATH:$JAVA_HOME/lib:$JAVA_HOME/jre/lib
ENV?PATH=$JAVA_HOME/bin:$JAVA_HOME/jre/bin:$PATH:$HOME/bin
RUN?groupadd?elsearch?&&?\
useradd?elsearch?-g?elsearch?-p?elasticsearch?&&?\
chown?-R?elsearch:elsearch?/usr/local/elasticsearch-6.6.1?&&?\
cp?/usr/share/zoneinfo/Asia/Shanghai?/etc/localtime?&&?\
echo?"Asia/shanghai"?>?/etc/timezone?&&?\
yum?install?which?-y?&&?\
mkdir?/opt/data?&&?\
mkdir?/opt/logs
EXPOSE?9200?9300
#主要是切換到elsearch用戶啟動es
USER?elsearch
WORKDIR?/usr/local/elasticsearch-6.6.1/bin/
ENTRYPOINT?["./elasticsearch"]

2、elasticsearch.yml

[root@localhost?elasticsearch]#?egrep??"^[^#]"?elasticsearch.yml?
cluster.name:?es-cluster
node.name:?node-1
path.data:?/opt/data
path.logs:?/opt/logs
network.host:?0.0.0.0
http.port:?9200
cluster.routing.allocation.disk.threshold_enabled:?true
cluster.routing.allocation.disk.watermark.low:?94%
cluster.routing.allocation.disk.watermark.high:?96%
cluster.routing.allocation.disk.watermark.flood_stage:?98%
discovery.zen.minimum_master_nodes:?1

● logstash

1、logstash-dockerfile

FROM?centos:latest
ADD?logstash-6.6.1.tar.gz?/usr/local/
COPY?logstash.yml?/usr/local/logstash-6.6.1/config/
COPY?logstash.conf?/usr/local/logstash-6.6.1/config/
COPY?jdk1.8?/usr/local/
COPY?start.sh?/start.sh
ENV?JAVA_HOME=/usr/local/jdk1.8
ENV?CLASSPATH=$CLASSPATH:$JAVA_HOME/lib:$JAVA_HOME/jre/lib
ENV?PATH=$JAVA_HOME/bin:$JAVA_HOME/jre/bin:$PATH:$HOME/bin
RUN?mkdir?/opt/data?&&?\
mkdir?/opt/logs?&&?\
chmod?+x?/start.sh
ENTRYPOINT?["/start.sh"]

2、logstash-start.sh

#!/bin/bash
/usr/local/logstash-6.6.1/bin/logstash?-f?/usr/local/logstash-6.6.1/config/logstash.conf

3、logstash.yml

[root@localhost?logstash]#?egrep?"^[^#]"?logstash.yml?
path.data:?/opt/data
path.logs:?/opt/logs
pipeline.batch.size:?200

4、logstash.conf

input?{
??file?{
????path?=>?"/usr/local/nginx/logs/access.log"
????type?=>?"nginx"
????start_position?=>?"beginning"
????sincedb_path?=>?"/dev/null"
??}
??file?{
????path?=>?"/var/log/secure"
????type?=>?"secure"
????start_position?=>?"beginning"
????sincedb_path?=>?"/dev/null"
??}
}
#詳細說明可以查看我之前的博客
filter?{
????grok?{
????????match?=>?{
????????????"message"?=>?'(?<clientip>[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3})?-?-?(?<requesttime>\[[0-9]{1,2}\/[A-z]+\/[0-9]{4}\:[0-9]{2}\:[0-9]{2}\:[0-9]{2}?\+[0-9]*\])?"(?<requesttype>[A-Z]+)?(?<requesturl>[^?]+)?(?<requestv>HTTP/\d\.\d)"?(?<requestnode>[0-9]+)?(?<requestsize>[0-9]+)?"(?<content>[^?]|(http|https)://[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}\/)"?"(?<ua>(a-Z|0-9|?|.)+)"'
????????}
?????????remove_field?=>?["message","log","beat","offset","prospector","host","@version"]
????}
}
#output指向es容器
output?{
?if?[type]?==?"nginx"?{
??elasticsearch?{
????hosts?=>?["es:9200"]
????index?=>?"nginx-%{+YYYY.MM.dd}"
????????}
?????}
??else?if?[type]?==?"secure"?{
????elasticsearch?{
????hosts?=>?["es:9200"]
????index?=>?"secure-%{+YYYY.MM.dd}"
????????}
?????}
??}

● kibana

1、kibana-dockerfile

FROM?centos:latest
ADD?kibana-6.6.1-linux-x86_64.tar.gz???/usr/local/
COPY?kibana.yml?/usr/local/kibana-6.6.1-linux-x86_64/config/
COPY?start.sh?/start.sh
RUN??chmod?+x?/start.sh
EXPOSE?5601
ENTRYPOINT?["/start.sh"]

2、kibana.yml

[root@localhost?kibana]#?egrep?"^[^#]"?kibana.yml?
server.port:?5601
server.host:?"0.0.0.0"
#指向es容器的9200端口
elasticsearch.hosts:?["http://es:9200"]

3、kibana-start.sh

#!/bin/bash
/usr/local/kibana-6.6.1-linux-x86_64/bin/kibana

二、docker-compose,yml文件編寫

[root@localhost elk_dockerfile]# cat docker-compose.yml?

version:?'3.7'
services:
??elasticsearch:
????image:?elasticsearch:elk
????container_name:?es
????networks:
??????-?elk
????volumes:
??????-?/opt/data:/opt/data
??????-?/opt/logs:/opt/logs
????expose:
??????-?9200
??????-?9300
????restart:?always
????depends_on:
??????-?logstash
??????-?kibana
??logstash:
????image:?logstash:elk
????container_name:?logstash
????networks:
??????-?elk
????volumes:
??????-?/opt/logstash/data/:/op/data
??????-?/opt/logstash/logs/:/opt/logs
??????-?/opt/elk/elk_dockerfile/logstash/logstash.conf:/usr/local/logstash-6.6.1/config/logstash.conf
??????-?/usr/local/nginx/logs:/usr/local/nginx/logs
??????-?/var/log/secure:/var/log/secure
????restart:?always
??kibana:
????image:?kibana:elk
????container_name:?kibana
????ports:
??????-?5601:5601
????networks:
??????-?elk
????volumes:
??????-?/opt/elk/elk_dockerfile/kibana/kibana.yml:/usr/local/kibana-6.6.1-linux-x86_64/config/kibana.yml
networks:
??elk:

compose文件version版本指向

三、訪問界面