shell腳本編程之正則表達(dá)式（二）（擴(kuò)展正則表達(dá)式、sed）

shell腳本編程之正則表達(dá)式（二）

一、前言

? 本文主要是對(duì)擴(kuò)展正則表達(dá)式的介紹，同時(shí)，繼續(xù)按照上篇文章的風(fēng)格介紹sed文本處理工具，sed作為shell編程中“三劍客”之一，在對(duì)文本處理上有巨大作用。
關(guān)于正則概念以及grep命令結(jié)合正則使用的案例請(qǐng)參照：https://blog.51cto.com/14557673/2455588

二、擴(kuò)展正則表達(dá)式

? 擴(kuò)展正則表達(dá)式主要是為了簡(jiǎn)化指令而產(chǎn)出的。例如，使用基礎(chǔ)正則表達(dá)式查詢文件中空白行與行首為#號(hào)之外的行（一般用于查看生效的配置），執(zhí)行"grep -v '^$' test.txt | grep -v '^#",而使用擴(kuò)展正則表達(dá)式，可以簡(jiǎn)化為“egrep -v '^$|^#' test.txt”，由于grep僅支持基礎(chǔ)正則表達(dá)式，如果想要使用擴(kuò)展正則表達(dá)式需要使用egrep或者awk命令。下一篇會(huì)講解awk命令。

? egrep 命令是一個(gè)搜索文件獲得模式，使用該命令可以搜索文件中的任意字符串和符號(hào)，也可以搜索一個(gè)或多個(gè)文件的字符串，一個(gè)提示符可以是單個(gè)字符、一個(gè)字符串、一個(gè)字或一個(gè)句子。

? 擴(kuò)展正則表達(dá)式常見元字符表

三、文本處理器之——sed工具簡(jiǎn)介

sed （Stream Editor）是一個(gè)強(qiáng)大而簡(jiǎn)單的文本解析轉(zhuǎn)換工具，可以讀取文本，并根據(jù)指定的條件對(duì)文本內(nèi)容進(jìn)行編輯（刪除、替換、添加、移動(dòng)等），最后輸出所有行或者僅輸出處理的某些行。sed 也可以在無(wú)交互的情況下實(shí)現(xiàn)相當(dāng)復(fù)雜的文本處理操作，被廣泛應(yīng)用于 Shell 腳本中，用以完成各種自動(dòng)化處理任務(wù)。

? sed 的工作流程主要包括讀取、執(zhí)行和顯示三個(gè)過(guò)程。

讀?。簊ed 讀取的是行內(nèi)容并且將其存儲(chǔ)到臨時(shí)的緩沖區(qū)中（或稱為模式空間，pattern space）

執(zhí)行：默認(rèn)在模式空間中順序執(zhí)行，除非指定了行的地址。

顯示：發(fā)送修改后的內(nèi)容到輸出流。再發(fā)送數(shù)據(jù)后，模式空間將會(huì)被清空。

注意：默認(rèn)情況下,所有的 sed 命令都是在模式空間內(nèi)執(zhí)行的，因此輸入的文件并不會(huì)發(fā)生任何變化，除非是用重定向存儲(chǔ)輸出。

用法：sed [選項(xiàng)] ’操作‘ 參數(shù)或者sed [選項(xiàng)] -f scriptfile 參數(shù)

選項(xiàng)：

• -e 或--expression=：表示用指定命令或者腳本來(lái)處理輸入的文本文件。
• -f 或--file=：表示用指定的腳本文件來(lái)處理輸入的文本文件。
• -h 或--help：顯示幫助。
• -n、--quiet 或 silent：表示僅顯示處理后的結(jié)果。
• -i：直接編輯文本文件。

操作：

• a：增加，在當(dāng)前行下面增加一行指定內(nèi)容。
• c：替換，將選定行替換為指定內(nèi)容。
• d：刪除，刪除選定的行。
• i：插入，在選定行上面插入一行指定內(nèi)容。
• p：打印，如果同時(shí)指定行，表示打印指定行；如果不指定行，則表示打印所有內(nèi)容；如果有非打印字符，則以 ASCII 碼輸出。其通常與“-n”選項(xiàng)一起使用。
• s：替換，替換指定字符。
• y：字符轉(zhuǎn)換。

四、sed用法實(shí)例細(xì)講

1）輸出符合條件的文本（p表示正常輸出）

[root@lokott opt]# sed -n 'p' test.txt   //相當(dāng)于cat test.txt
he was short and fat.
He was wearing a blue polo shirt with black pants. 
The home of Football on BBC Sport online.
the tongue is boneless but it breaks bones.12!
 google is the best tools for search keyword.
The year ahead will test our political establishment to the limit.
PI=3.141592653589793238462643383249901429
a wood cross!
Actions speak louder than words

#woood #
#woooooood # 
AxyzxyzxyzxyzC
I bet this place is really spooky late at night! 
Misfortunes never come alone/single.
I shouldn't have lett so tast.
[root@lokott opt]# cat test.txt |wc -l
17
[root@lokott opt]# sed -n 'p' test.txt |wc -l
17

[root@lokott opt]# sed -n '2p' test.txt                 //顯示第二行
He was wearing a blue polo shirt with black pants. 
[root@lokott opt]# sed -n '2,5p' test.txt               //顯示2-5行
He was wearing a blue polo shirt with black pants. 
The home of Football on BBC Sport online.
the tongue is boneless but it breaks bones.12!
google is the best tools for search keyword.

[root@lokott opt]# sed -n 'n;p' test.txt       //循環(huán)顯示，跳過(guò)一行，再顯示一行，以此類推
He was wearing a blue polo shirt with black pants. 
the tongue is boneless but it breaks bones.12!
The year ahead will test our political establishment to the limit.
a wood cross!

#woood #
AxyzxyzxyzxyzC
Misfortunes never come alone/single.
[root@lokott opt]# sed -n 'p;n' test.txt      //循環(huán)顯示，先顯示一行，再跳過(guò)一行，一次類推
he was short and fat.
The home of Football on BBC Sport online.
 google is the best tools for search keyword.
PI=3.141592653589793238462643383249901429
Actions speak louder than words

#woooooood # 
I bet this place is really spooky late at night! 
I shouldn't have lett so tast.

[root@lokott opt]# nl test.txt  //顯示行號(hào)并且列出文本內(nèi)容，這里為了方便演示，其行號(hào)不等同實(shí)際操作的行號(hào)
     1  he was short and fat.               
     2  He was wearing a blue polo shirt with black pants. 
     3  The home of Football on BBC Sport online.
     4  the tongue is boneless but it breaks bones.12!
     5   google is the best tools for search keyword.
     6  The year ahead will test our political establishment to the limit.
     7  PI=3.141592653589793238462643383249901429
     8  a wood cross!
     9  Actions speak louder than words

    10  #woood #
    11  #woooooood # 
    12  AxyzxyzxyzxyzC
    13  I bet this place is really spooky late at night! 
    14  Misfortunes never come alone/single.
    15  I shouldn't have lett so tast.
[root@lokott opt]# sed -n '2,5{p;n}' test.txt   顯示2和4行
He was wearing a blue polo shirt with black pants. 
the tongue is boneless but it breaks bones.12!
[root@lokott opt]# sed -n '2,${p;n}' test.txt  //2到末行隔行顯示
He was wearing a blue polo shirt with black pants. 
the tongue is boneless but it breaks bones.12!
The year ahead will test our political establishment to the limit.
a wood cross!

#woood #
AxyzxyzxyzxyzC
Misfortunes never come alone/single.

sed結(jié)合正則表達(dá)式使用時(shí)，格式略有不同，正則表達(dá)式須以“/”包圍，實(shí)例如下：

[root@lokott opt]# sed -n '/the/p' test.txt                     //輸出包含the的行
the tongue is boneless but it breaks bones.12!
 google is the best tools for search keyword.
The year ahead will test our political establishment to the limit. 
[root@lokott opt]# sed -n '4,/the/p' test.txt  //這里無(wú)論第四行是否有the都會(huì)講其顯示，因?yàn)樵摋l命令的含
the tongue is boneless but it breaks bones.12!  //義是從第四行顯示直到遇到第一個(gè)the的所有內(nèi)容，第四行
 google is the best tools for search keyword.   //的the不算
[root@lokott opt]# sed -n '4,/the/p' test.txt  //更改了文本演示效果
tahe tongue is boneless but it breaks bones.12!
google is the best tools for search keyword.
[root@lokott opt]# sed -n '/the/=' test.txt //顯示包含the的行號(hào)
5
6
[root@lokott opt]# sed -n '/^PI/p' test.txt //顯示以PI開頭的行  可以用grep '^PI' test.txt代替
PI=3.141592653589793238462643383249901429
[root@lokott opt]# sed -n '/[0-9]$/p' test.txt //顯示以數(shù)字結(jié)尾的行
PI=3.141592653589793238462643383249901429
[root@lokott opt]# sed -n '/\<wood\>/p' test.txt //輸出包含單詞wood 的行,\< \>代表單詞邊界
a wood cross!

2）刪除符合條件的文本（d）

[root@lokott opt]# nl test.txt | sed '3d' //刪除第三行
     1  he was short and fat.
     2  He was wearing a blue polo shirt with black pants. 
     4  tahe tongue is boneless but it breaks bones.12!
     5  google is the best tools for search keyword.
     6  The year ahead will test our political establishment to the limit.
     7  PI=3.141592653589793238462643383249901429
     8  a wood cross!
     9  Actions speak louder than words

    10  #woood #
    11  #woooooood # 
    12  AxyzxyzxyzxyzC
    13  I bet this place is really spooky late at night! 
    14  Misfortunes never come alone/single.
    15  I shouldn't have lett so tast.

[root@lokott opt]# nl test.txt | sed '3,5d' //刪除3-5行
     1  he was short and fat.
     2  He was wearing a blue polo shirt with black pants. 
     6  The year ahead will test our political establishment to the limit.
     7  PI=3.141592653589793238462643383249901429
     8  a wood cross!
     9  Actions speak louder than words

    10  #woood #
    11  #woooooood # 
    12  AxyzxyzxyzxyzC
    13  I bet this place is really spooky late at night! 
    14  Misfortunes never come alone/single.
    15  I shouldn't have lett so tast.
[root@lokott opt]# nl test.txt | sed '/cross/d' //刪除含有cross的行
     1  he was short and fat.
     2  He was wearing a blue polo shirt with black pants. 
     3  The home of Football on BBC Sport online.
     4  tahe tongue is boneless but it breaks bones.12!
     5  google is the best tools for search keyword.
     6  The year ahead will test our political establishment to the limit.
     7  PI=3.141592653589793238462643383249901429
     9  Actions speak louder than words

    10  #woood #
    11  #woooooood # 
    12  AxyzxyzxyzxyzC
    13  I bet this place is really spooky late at night! 
    14  Misfortunes never come alone/single.
    15  I shouldn't have lett so tast.

[root@lokott opt]# sed '/^[a-z]/d' test.txt | nl  //刪除所有小寫字母開頭的行
     1  He was wearing a blue polo shirt with black pants. 
     2  The home of Football on BBC Sport online.
     3  The year ahead will test our political establishment to the limit.
     4  PI=3.141592653589793238462643383249901429
     5  Actions speak louder than words

     6  #woood #
     7  #woooooood # 
     8  AxyzxyzxyzxyzC
     9  I bet this place is really spooky late at night! 
    10  Misfortunes never come alone/single.
    11  I shouldn't have lett so tast.

[root@lokott opt]# sed '/\.$/d' test.txt   //刪除以點(diǎn)為結(jié)尾的行
He was wearing a blue polo shirt with black pants. 
tahe tongue is boneless but it breaks bones.12!
PI=3.141592653589793238462643383249901429
a wood cross!
Actions speak louder than words

#woood #
#woooooood # 
AxyzxyzxyzxyzC
I bet this place is really spooky late at night!

[root@lokott opt]# sed '/^$/d' test.txt             //刪除所有空行 
he was short and fat.
He was wearing a blue polo shirt with black pants. 
The home of Football on BBC Sport online.
tahe tongue is boneless but it breaks bones.12!
google is the best tools for search keyword.
The year ahead will test our political establishment to the limit.
PI=3.141592653589793238462643383249901429
a wood cross!
Actions speak louder than words
#woood #
#woooooood # 
AxyzxyzxyzxyzC
I bet this place is really spooky late at night! 
Misfortunes never come alone/single.
I shouldn't have lett so tast.

3）替換符合條件的文本

? 在使用 sed 命令進(jìn)行替換操作時(shí)需要用到 s（字符串替換）、c（整行/整塊替換）、y（字符轉(zhuǎn)換）命令選項(xiàng)，常見的用法如下所示。

sed 's/the/THE/' test.txt //將每行中的第一個(gè)the 替換為 THE

sed 's/l/L/2' test.txt //將每行中的第 2 個(gè)l 替換為L(zhǎng)

sed 's/the/THE/g' test.txt //將文件中的所有the 替換為THE

sed 's/o//g**' test.txt** //將文件中的所有o 刪除(替換為空串)

sed 's/^/#/**' test.txt** //在每行行首插入#號(hào)

sed '/the/s/^/#/' test.txt //在包含the 的每行行首插入#號(hào)

sed 's/$/EOF/' test.txt //在每行行尾插入字符串EOF

sed '3,5s/the/THE/g' test.txt //將第 3~5 行中的所有the 替換為 THE

sed '/the/s/o/O/g**' test.txt** //將包含the 的所有行中的o 都替換為 O

4） 遷移符合條件的文本

其中，H，復(fù)制到剪貼板；g、G，將剪貼板中的數(shù)據(jù)覆蓋/追加至指定行；w，保存為文件；r，讀取指定文件；a，追加指定內(nèi)容。

sed '/the/{H;d};$G' test.txt //將包含the 的行遷移至文件末尾,{;}用于多個(gè)操作

sed '1,5{H;d};17G' test.txt //將第 1~5 行內(nèi)容轉(zhuǎn)移至第 17 行后

sed '/the/w out.file' test.txt //將包含the 的行另存為文件out.file

sed '/the/r /etc/hostname' test.txt //將文件/etc/hostname 的內(nèi)容添加到包含the 的每行以后

sed '3aNew' test.txt //在第 3 行后插入一個(gè)新行,內(nèi)容為 New

sed '/the/aNew' test.txt //在包含the 的每行后插入一個(gè)新行,內(nèi)容為 New

sed '3aNew1\nNew2' test.txt //在第 3 行后插入多行內(nèi)容,中間的\n 表示換行

未完待續(xù).......)