溫馨提示×

溫馨提示×

您好,登錄后才能下訂單哦!

密碼登錄×
登錄注冊×
其他方式登錄
點擊 登錄注冊 即表示同意《億速云用戶服務(wù)條款》

Apache服務(wù)的日志分割與日志分析

發(fā)布時間:2020-07-23 06:00:16 來源:網(wǎng)絡(luò) 閱讀:429 作者:wx5d2c2d660c282 欄目:系統(tǒng)運維

日志分割

  • 隨著網(wǎng)站的訪問量增加,默認(rèn)情況下Apache的單個日志文件也會越來越大

  • 日志文件占用磁盤空間很大

  • 查看相關(guān)信息不方便

兩種日志分割工具

  • Apache自帶rotatelogs分割工具實現(xiàn)

  • 第三方工具cronolog分割

rotatelogs分割工具實驗

(1)安裝httpd服務(wù)。

[root@localhost ~]# yum install httpd -y
........省略安裝過程

Apache服務(wù)的日志分割與日志分析

(2)安裝完畢后,我們可以查看,日志文件存放目錄“/var/log/httpd/”可以看到并沒有日志文件。當(dāng)我們開啟服務(wù)后再次查看,日志文件生成。

[root@localhost ~]# ls /var/log/httpd/                        //查看目錄
[root@localhost ~]# systemctl start httpd                //開啟服務(wù)
[root@localhost ~]# ls /var/log/httpd/                       //查看目錄
access_log  error_log                                               //日志文件
[root@localhost ~]# systemctl stop firewalld.service                //關(guān)閉防火墻
[root@localhost ~]# setenforce 0                           //關(guān)閉增強性安全功能
[root@localhost ~]#

(3)查看IP地址。

[root@localhost ~]# ifconfig 
ens33: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
        inet 192.168.52.133  netmask 255.255.255.0  broadcast 192.168.52.255
        inet6 fe80::3e1d:31ba:f66a:6f80  prefixlen 64  scopeid 0x20<link>
        ether 00:0c:29:27:1c:3f  txqueuelen 1000  (Ethernet)
        RX packets 11004  bytes 14155417 (13.4 MiB)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 5374  bytes 360388 (351.9 KiB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

(4)對配置文件進(jìn)行修改(可以用“ / ”來查詢關(guān)鍵字修改)。

[root@localhost ~]# vim /etc/httpd/conf/httpd.conf

Listen 192.168.52.133:80          //開啟IPv4監(jiān)聽
#Listen 80                                    //注釋IPv6監(jiān)聽

ServerName www.abc.com:80          //設(shè)置域名

#ErrorLog "logs/error_log"
ErrorLog "| /usr/sbin/rotatelogs -l logs/www.abc.com.error_%Y%m%dlog 86400"   //錯誤日志配置

# CustomLog "logs/access_log" combined
CustomLog "| /usr/sbin/rotatelogs -l logs/www.abc.com.access_%Y%m%dlog 86400" combined   //訪問日志配置

[root@localhost ~]# systemctl restart httpd                //重啟服務(wù)
[root@localhost ~]#

Apache服務(wù)的日志分割與日志分析

Apache服務(wù)的日志分割與日志分析

Apache服務(wù)的日志分割與日志分析

(5)我們先查看日志文件目錄,此時只有錯誤日志。當(dāng)我們用win10主機訪問網(wǎng)站后再查看,就生成了訪問日志。

Apache服務(wù)的日志分割與日志分析

[root@localhost ~]# 
[root@localhost ~]# cd /var/log/httpd/                //進(jìn)入日志文件存放目錄
[root@localhost httpd]# ls                  //查看
access_log  error_log  www.abc.com.error_20191025log        //只有錯誤日志
[root@localhost httpd]# ls                 //用win10主機訪問后再次查看
access_log  error_log  www.abc.com.access_20191025log  www.abc.com.error_20191025log   //訪問日志生成
[root@localhost httpd]# 

(6)我們用“date -s”命令,將系統(tǒng)日期向后修改一天后,重啟服務(wù)再次查看目錄,又生成了新的一天的日志文件。

[root@localhost httpd]# date -s 10/26/19
2019年 10月 26日 星期六 00:00:00 CST
[root@localhost httpd]# 
[root@localhost httpd]# ls
access_log  error_log  www.abc.com.access_20191025log  www.abc.com.error_20191025log
[root@localhost httpd]# systemctl restart httpd
[root@localhost httpd]# ls
access_log  error_log  www.abc.com.access_20191025log  www.abc.com.error_20191025log  www.abc.com.error_20191026log
[root@localhost httpd]# 

第三方工具cronolog實驗

(1)我們先安裝httpd服務(wù)。

[root@localhost ~]# yum install httpd -y
......//省略安裝過程
[root@localhost ~]# 

(2)通過Samba服務(wù)將cronolog工具的安裝包從宿主機掛載到Linux系統(tǒng)。

[root@localhost ~]# mkdir /mnt/tools             //創(chuàng)建掛載目錄
[root@localhost ~]# smbclient -L //192.168.100.50/      //檢查共享
Enter SAMBA\root's password: 
OS=[Windows 10 Enterprise LTSC 2019 17763] Server=[Windows 10 Enterprise LTSC 2019 6.3]

    Sharename       Type      Comment
    ---------       ----      -------
    IPC$            IPC       遠(yuǎn)程 IPC
    share           Disk      
    tools           Disk      
    Users           Disk      
Connection to 192.168.100.50 failed (Error NT_STATUS_RESOURCE_NAME_NOT_FOUND)
NetBIOS over TCP disabled -- no workgroup available
[root@localhost ~]# mount.cifs //192.168.100.50/tools /mnt/tools           //掛載
Password for root@//192.168.100.50/tools:  
[root@localhost ~]# ls /mnt/tools/          //查看
awstats-7.6.tar.gz                fiddler.exe                john-1.8.0.tar.gz
cronolog-1.6.2-14.el7.x86_64.rpm  intellijideahahau2018.rar  LAMP-C7
extundelete-0.2.4.tar.bz2         jdk-8u191-windows-x64.zip
[root@localhost ~]# 

(3)用“rpm”命令直接安裝cronolog工具。

[root@localhost ~]# cd /mnt/tools            //切換目錄
[root@localhost tools]# ls            //查看
awstats-7.6.tar.gz                fiddler.exe                john-1.8.0.tar.gz
cronolog-1.6.2-14.el7.x86_64.rpm  intellijideahahau2018.rar  LAMP-C7
extundelete-0.2.4.tar.bz2         jdk-8u191-windows-x64.zip
[root@localhost tools]# rpm -ivh cronolog-1.6.2-14.el7.x86_64.rpm        //安裝
警告:cronolog-1.6.2-14.el7.x86_64.rpm: 頭V3 RSA/SHA256 Signature, 密鑰 ID 352c64e5: NOKEY
準(zhǔn)備中...                          ################################# [100%]
正在升級/安裝...
   1:cronolog-1.6.2-14.el7            ################################# [100%]
[root@localhost tools]# 

(4) 查看命令文件存放目錄。

[root@localhost tools]# which cronolog    //查看命令文件存放目錄
/usr/sbin/cronolog
[root@localhost tools]# 

(5)對httpd服務(wù)的配置文件進(jìn)行修改。

[root@localhost tools]# vim /etc/httpd/conf/httpd.conf

Listen 192.168.52.133:80          //開啟IPv4監(jiān)聽
#Listen 80                                    //注釋IPv6監(jiān)聽

ServerName www.abc.com:80     //設(shè)置域名

#ErrorLog "logs/error_log"
ErrorLog "| /usr/sbin/cronolog logs/www.abc.com.error_%Y%m%d.log"   //配置錯誤日志

    #CustomLog "logs/access_log" combined
    CustomLog "| /usr/sbin/cronolog logs/www.abc.com.access_%Y%m%d.log" combined   //配置訪問日志

(6)實驗結(jié)果測試。

Apache服務(wù)的日志分割與日志分析

[root@localhost tools]# systemctl start httpd              //開啟httpd服務(wù)
[root@localhost tools]# systemctl stop firewalld.service           //關(guān)閉防火墻
[root@localhost tools]# setenforce 0          //關(guān)閉增強性安全功能
[root@localhost tools]# ls /var/log/httpd/      //查看日志存放目錄
www.abc.com.error_20191025.log              //只有錯誤日志
[root@localhost tools]# ls /var/log/httpd/       //用win10主機訪問后再查看
www.abc.com.access_20191025.log  www.abc.com.error_20191025.log    //訪問日志生成
[root@localhost tools]# 
[root@localhost tools]# date -s 10/26/19     //更改系統(tǒng)時間
2019年 10月 26日 星期六 00:00:00 CST
[root@localhost tools]# systemctl restart httpd      //重啟服務(wù)
[root@localhost tools]# ls /var/log/httpd/               //再次查看日志存放目錄
www.abc.com.access_20191025.log  www.abc.com.error_20191025.log  www.abc.com.error_20191026.log    //生成新日志文件
[root@localhost tools]# 

AWStats日志分析系統(tǒng)介紹

AWStats 日志分析系統(tǒng)

  • Per語言開發(fā)的一款開源日志分析系統(tǒng)

  • 可用來分析Apache、Samba、 Vsftpd、 IIS等服務(wù)器的訪問日志信息結(jié)合crond等計劃任務(wù)服務(wù),可對日志內(nèi)容定期進(jìn)行分析

部署AWStats日志分析系統(tǒng)實驗

(1)安裝httpd服務(wù)和bind(named)服務(wù)。

[root@localhost ~]# yum install bind httpd -y
......//省略安裝過程

Apache服務(wù)的日志分割與日志分析

(2)對named服務(wù)的主配置文件進(jìn)行修改。

[root@localhost ~]# vim /etc/named.conf

options {
        listen-on port 53 { any; };                   //127.0.0.1改為any
        listen-on-v6 port 53 { ::1; };
        directory       "/var/named";
        dump-file       "/var/named/data/cache_dump.db";
        statistics-file "/var/named/data/named_stats.txt";
        memstatistics-file "/var/named/data/named_mem_stats.txt";
        recursing-file  "/var/named/data/named.recursing";
        secroots-file   "/var/named/data/named.secroots";
        allow-query     { any; };               //localhost改為any

(3)對named服務(wù)的區(qū)域配置文件進(jìn)行修改。

[root@localhost ~]# vim /etc/named.rfc1912.zones

zone "abc.com" IN {                                 //添加一個域名信息
        type master;
        file "abc.com.zone";
        allow-update { none; };
};

(4)查看一下IP地址。

[root@localhost named]# ifconfig 
ens33: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
        inet 192.168.52.133  netmask 255.255.255.0  broadcast 192.168.52.255
        inet6 fe80::3e1d:31ba:f66a:6f80  prefixlen 64  scopeid 0x20<link>
        ether 00:0c:29:27:1c:3f  txqueuelen 1000  (Ethernet)
        RX packets 14532  bytes 20210558 (19.2 MiB)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 6054  bytes 399142 (389.7 KiB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

(5)保留權(quán)限復(fù)制一份named服務(wù)的區(qū)域數(shù)據(jù)配置文件,進(jìn)行修改。

[root@localhost ~]# cd /var/named/          //切換目錄
[root@localhost named]# ls                        //查看
data  dynamic  named.ca  named.empty  named.localhost  named.loopback  slaves
[root@localhost named]# cp -p named.localhost abc.com.zone    //復(fù)制
[root@localhost named]# vim abc.com.zone 

$TTL 1D
@       IN SOA  @ rname.invalid. (
                                        0       ; serial
                                        1D      ; refresh
                                        1H      ; retry
                                        1W      ; expire
                                        3H )    ; minimum
        NS      @
        A       127.0.0.1
www IN  A       192.168.52.133             //添加IPv4的域名解析

(6)開啟named服務(wù)。

[root@localhost named]# systemctl start named      //開啟服務(wù)
[root@localhost named]# systemctl stop firewalld.service      //關(guān)閉防火墻
[root@localhost named]# setenforce 0         //關(guān)閉增強性安全功能
[root@localhost named]# 

(7)對httpd服務(wù)的配置文件進(jìn)行修改。

[root@localhost named]# cd /etc/httpd/
[root@localhost httpd]# ls
conf  conf.d  conf.modules.d  logs  modules  run
[root@localhost httpd]# vim conf/httpd.conf

Listen 192.168.52.133:80      //開啟IPv4監(jiān)聽
#Listen 80                               //注銷IPv6監(jiān)聽

#
ServerName www.abc.com:80     //配置域名

(8)開啟服務(wù),查看日志文件存放目錄

[root@localhost httpd]# systemctl start httpd.service    //開啟服務(wù)
[root@localhost httpd]# 
[root@localhost httpd]# ls /var/log/httpd/      //查看
access_log  error_log       //訪問日志和錯誤日志文件
[root@localhost httpd]# 

(9)用Samba服務(wù)將工具壓縮包,從宿主機掛載到Linux系統(tǒng)。

[root@localhost httpd]# mkdir /mnt/abc     //創(chuàng)建掛載目錄
[root@localhost httpd]# smbclient -L //192.168.100.50/      //查看共享
Enter SAMBA\root's password: 
OS=[Windows 10 Enterprise LTSC 2019 17763] Server=[Windows 10 Enterprise LTSC 2019 6.3]

    Sharename       Type      Comment
    ---------       ----      -------
    IPC$            IPC       遠(yuǎn)程 IPC
    share           Disk      
    tools           Disk      
    Users           Disk      
Connection to 192.168.100.50 failed (Error NT_STATUS_RESOURCE_NAME_NOT_FOUND)
NetBIOS over TCP disabled -- no workgroup available
[root@localhost httpd]# mount.cifs //192.168.100.50/tools /mnt/abc      //掛載
Password for root@//192.168.100.50/tools:  
[root@localhost httpd]# ls /mnt/abc/      //查看掛載目錄
awstats-7.6.tar.gz                fiddler.exe                john-1.8.0.tar.gz
cronolog-1.6.2-14.el7.x86_64.rpm  intellijideahahau2018.rar  LAMP-C7
extundelete-0.2.4.tar.bz2         jdk-8u191-windows-x64.zip
[root@localhost httpd]# tar zxvf /mnt/abc/awstats-7.6.tar.gz -C /opt/      //將壓縮包解壓到“/opt/”目錄
[root@localhost httpd]# ls /opt      //查看“/opt/”目錄
awstats-7.6  rh
[root@localhost httpd]#
[root@localhost httpd]# mv /opt/awstats-7.6/ /usr/local/awstats      //將解壓包移動到“/usr/local/”目錄,命名為“awstats ”
[root@localhost httpd]# ls /usr/local/     //查看
awstats  bin  etc  games  include  lib  lib64  libexec  sbin  share  src
[root@localhost httpd]# 

(10)進(jìn)入“/usr/local/awstats/tools”目錄

[root@localhost httpd]# 
[root@localhost httpd]# cd /usr/local/awstats/
[root@localhost awstats]# ls
docs  README.md  tools  wwwroot
[root@localhost awstats]# cd tools/
[root@localhost tools]# ls
awstats_buildstaticpages.pl  dolibarr            maillogconvert.pl   xslt
awstats_configure.pl         geoip_generator.pl  nginx
awstats_exportlib.pl         httpd_conf          urlaliasbuilder.pl
awstats_updateall.pl         logresolvemerge.pl  webmin

(11)對awstats進(jìn)行配置。

[root@localhost tools]# ./awstats_configure.pl 

----- AWStats awstats_configure 1.0 (build 20140126) (c) Laurent Destailleur -----
This tool will help you to configure AWStats to analyze statistics for
one web server. You can try to use it to let it do all that is possible
in AWStats setup, however following the step by step manual setup
documentation (docs/index.html) is often a better idea. Above all if:
- You are not an administrator user,
- You want to analyze downloaded log files without web server,
- You want to analyze mail or ftp log files instead of web log files,
- You need to analyze load balanced servers log files,
- You want to 'understand' all possible ways to use AWStats...
Read the AWStats documentation (docs/index.html).

-----> Running OS detected: Linux, BSD or Unix

-----> Check for web server install

Enter full config file path of your Web server.
Example: /etc/httpd/httpd.conf
Example: /usr/local/apache2/conf/httpd.conf
Example: c:\Program files\apache group\apache\conf\httpd.conf
Config file path ('none' to skip web server setup):
> /etc/httpd/conf/httpd.conf                      //指定httpd服務(wù)配置文件

-----> Check and complete web server config file '/etc/httpd/conf/httpd.conf'
  Add 'Alias /awstatsclasses "/usr/local/awstats/wwwroot/classes/"'
  Add 'Alias /awstatscss "/usr/local/awstats/wwwroot/css/"'
  Add 'Alias /awstatsicons "/usr/local/awstats/wwwroot/icon/"'
  Add 'ScriptAlias /awstats/ "/usr/local/awstats/wwwroot/cgi-bin/"'
  Add '<Directory>' directive
  AWStats directives added to Apache config file.

-----> Update model config file '/usr/local/awstats/wwwroot/cgi-bin/awstats.model.conf'
  File awstats.model.conf updated.

-----> Need to create a new config file ?
Do you want me to build a new AWStats config/profile
file (required if first install) [y/N] ? y                            //確定

-----> Define config file name to create
What is the name of your web site or profile analysis ?
Example: www.mysite.com
Example: demo
Your web site, virtual server or profile name:
> www.abc.com                                       //指定域名

-----> Define config file path
In which directory do you plan to store your config file(s) ?
Default: /etc/awstats
Directory path to store config file(s) (Enter for default):
>                //指定awstats工具配置文件,直接回車默認(rèn)“/etc/awstats”

-----> Create config file '/etc/awstats/awstats.www.abc.com.conf'
 Config file /etc/awstats/awstats.www.abc.com.conf created.

-----> Restart Web server with '/sbin/service httpd restart'
Redirecting to /bin/systemctl restart httpd.service

-----> Add update process inside a scheduler
Sorry, configure.pl does not support automatic add to cron yet.
You can do it manually by adding the following command to your cron:
/usr/local/awstats/wwwroot/cgi-bin/awstats.pl -update -config=www.abc.com
Or if you have several config files and prefer having only one command:
/usr/local/awstats/tools/awstats_updateall.pl now
Press ENTER to continue...                     //回車?yán)^續(xù)

A SIMPLE config file has been created: /etc/awstats/awstats.www.abc.com.conf
You should have a look inside to check and change manually main parameters.
You can then manually update your statistics for 'www.abc.com' with command:
> perl awstats.pl -update -config=www.abc.com
You can also read your statistics for 'www.abc.com' with URL:
> http://localhost/awstats/awstats.pl?config=www.abc.com

Press ENTER to finish...              //回車完成

[root@localhost tools]# 

(12)對httpd服務(wù)的配置文件進(jìn)行修改。

[root@localhost tools]# cd /etc/httpd/
[root@localhost httpd]# ls
conf  conf.d  conf.modules.d  logs  modules  run
[root@localhost httpd]# vim conf/httpd.conf

#
# This is to permit URL access to scripts/files in AWStats directory.
#
<Directory "/usr/local/awstats/wwwroot">
    Options None
    AllowOverride None
    #Order allow,deny      //注釋
    #Allow from all           //注釋
    Require all granted      //添加
</Directory>

(13)對awstats日志分析系統(tǒng)的配置文件進(jìn)行修改

[root@localhost httpd]# 
[root@localhost httpd]# cd /etc/awstats/
[root@localhost awstats]# ls
awstats.www.abc.com.conf
[root@localhost awstats]# vim awstats.www.abc.com.conf

LogFile="/var/log/httpd/access_log"         //指定httpd訪問日志文件

DirData="/var/lib/awstats"                        //awstats文件,默認(rèn)不存在(需要創(chuàng)建)

(14)創(chuàng)建awstats文件,并重啟httpd服務(wù)。

[root@localhost awstats]# 
[root@localhost awstats]# ls /var/lib/awstats             //查看awstats文件不存在
ls: 無法訪問/var/lib/awstats: 沒有那個文件或目錄
[root@localhost awstats]# mkdir /var/lib/awstats      //創(chuàng)建awstats文件
[root@localhost awstats]# ls /var/lib/awstats
[root@localhost awstats]# 
[root@localhost awstats]# systemctl restart httpd.service    //重啟服務(wù)
[root@localhost awstats]# 

(15)修改win10主機的dns服務(wù)器IP地址,同時用win主機訪問httpd服務(wù)域名。

Apache服務(wù)的日志分割與日志分析

Apache服務(wù)的日志分割與日志分析

(16)用win10主機訪問awstats統(tǒng)計的主頁,域名:http://localhost/awstats/awstats.pl?config=www.abc.com。可以看到?jīng)]有統(tǒng)計信息,因為沒有更新數(shù)據(jù)。

Apache服務(wù)的日志分割與日志分析

(17)我們回到Linux系統(tǒng),切換到“/usr/local/awstats/tools/”,更新數(shù)據(jù)。

[root@localhost awstats]# cd /usr/local/awstats/tools/
[root@localhost tools]# ls
awstats_buildstaticpages.pl  awstats_exportlib.pl  dolibarr            httpd_conf          maillogconvert.pl  urlaliasbuilder.pl  xslt
awstats_configure.pl         awstats_updateall.pl  geoip_generator.pl  logresolvemerge.pl  nginx              webmin
[root@localhost tools]# ./awstats_updateall.pl now     //更新數(shù)據(jù)
Running '"/usr/local/awstats/wwwroot/cgi-bin/awstats.pl" -update -config=www.abc.com -configdir="/etc/awstats"' to update config www.abc.com
Create/Update database for config "/etc/awstats/awstats.www.abc.com.conf" by AWStats version 7.6 (build 20161204)
From data in log file "/var/log/httpd/access_log"...
Phase 1 : First bypass old records, searching new record...
Searching new records from beginning of log file...
Phase 2 : Now process new records (Flush history on disk after 20000 hosts)...
Jumped lines in file: 0
Parsed lines in file: 106
 Found 0 dropped records,
 Found 0 comments,
 Found 0 blank records,
 Found 1 corrupted records,
 Found 0 old records,
 Found 105 new qualified records.

[root@localhost tools]# 

(18)再次在win10 系統(tǒng)進(jìn)入統(tǒng)計頁面,已經(jīng)有了統(tǒng)計數(shù)據(jù)。

Apache服務(wù)的日志分割與日志分析

(19)為了方便使用awstats服務(wù)統(tǒng)計數(shù)據(jù),我們可以設(shè)置周期性任務(wù)。

[root@localhost tools]# crontab -e    //編輯周期性任務(wù)

*/5 * * * *     /usr/local/awstats/tools/awstats_updateall.pl now     //每五分鐘更新一次數(shù)據(jù)

[root@localhost tools]# 
您在 /var/spool/mail/root 中有郵件
[root@localhost tools]# 

(20)我們訪問數(shù)據(jù)統(tǒng)計頁面的域名“http://localhost/awstats/awstats.pl?config=www.abc.com”太長,不容易記憶。我們可以優(yōu)化網(wǎng)頁的地址,用“http://www.abc.com/aws.html”即可訪問。

[root@localhost tools]# 
[root@localhost tools]# cd /var/www/html/
[root@localhost html]# vim aws.html

<html>
 <head>
  <meta http-equiv=refresh content="0;url=http://www.abc.com/awstats/awstats.pl?config=www.abc.com">
 </head>
 <body></body>
</html>

[root@localhost html]# systemctl restart httpd.service      //重啟服務(wù)

(21)在win10主機瀏覽器輸入地址“http://www.abc.com/aws.html”,會自動轉(zhuǎn)到數(shù)據(jù)統(tǒng)計頁面。

Apache服務(wù)的日志分割與日志分析

向AI問一下細(xì)節(jié)

免責(zé)聲明:本站發(fā)布的內(nèi)容(圖片、視頻和文字)以原創(chuàng)、轉(zhuǎn)載和分享為主,文章觀點不代表本網(wǎng)站立場,如果涉及侵權(quán)請聯(lián)系站長郵箱:is@yisu.com進(jìn)行舉報,并提供相關(guān)證據(jù),一經(jīng)查實,將立刻刪除涉嫌侵權(quán)內(nèi)容。

AI