gawk gsub函數(shù)的實際應(yīng)用

本篇內(nèi)容主要講解“gawk gsub函數(shù)的實際應(yīng)用”，感興趣的朋友不妨來看看。本文介紹的方法操作簡單快捷，實用性強(qiáng)。下面就讓小編來帶大家學(xué)習(xí)“gawk gsub函數(shù)的實際應(yīng)用”吧!

在做一個數(shù)據(jù)清洗需求的時候，需要查詢兩張表里幾個字段相同的重復(fù)數(shù)據(jù)。大概思路就是用exists語句，類似：
select *
  from a
 where exists (select 1
          from b
         where a.col1 = b.col1
           and a.col2 = a.col2);
但是這里麻煩的地方在于要匹配的列太多了：
a.INFOCODE,a.SOURCENAME,a.SOURCETYPE,a.PUBLISHTYPE,a.NOTICEDATE,a.ENDDATE,a.NOTICETITLE,a.LANGUAGE,a.IMPORTLEVEL,a.SOURCEURL,a.ATTACHTYPE,a.ATTACHNAME,a.ATTACHSIZE,a.FORM,a.ACCESSORYNUM,a.NOTICESTATE,a.PUBLISHDATE,a.FILENUMBER
用Linux文本處理的方法解決這個問題：
先將這段放到一個文本里：
root@bd-dev-mingshuo-183:/tmp#more 1
a.INFOCODE,a.SOURCENAME,a.SOURCETYPE,a.PUBLISHTYPE,a.NOTICEDATE,a.ENDDATE,a.NOTICETITLE,a.LANGUAGE,a.IMPORTLEVEL,a.SOURCEURL,a.ATTACHTYPE,a.ATTACHNAME,a.ATTACHSIZE,a.FORM,a.ACCESSORYNUM,
a.NOTICESTATE,a.PUBLISHDATE,a.FILENUMBER 這里介紹一下gawk里的gsub函數(shù)  
gsub匹配所有的符合正則表達(dá)式的內(nèi)容，然后替換，相當(dāng)于 sed 's//g'  
語法如下：
gsub(regular expression, subsitution string, target string);
處理的目標(biāo)范圍是第三個字段，匹配條件是第一個參數(shù)，匹配后，替換為第二個參數(shù)。

將一行文本處理為多行文本：
root@bd-dev-mingshuo-183:/tmp#more 1|gawk 'gsub(/,/,"\n",$0)'
a.INFOCODE
a.SOURCENAME
a.SOURCETYPE
a.PUBLISHTYPE
a.NOTICEDATE
a.ENDDATE
a.NOTICETITLE
a.LANGUAGE
a.IMPORTLEVEL
a.SOURCEURL
a.ATTACHTYPE
a.ATTACHNAME
a.ATTACHSIZE
a.FORM
a.ACCESSORYNUM
a.NOTICESTATE
a.PUBLISHDATE
a.FILENUMBER 復(fù)制每一列：
root@bd-dev-mingshuo-183:/tmp#more 1|gawk 'gsub(/,/,"\n",$0)'|gawk -F'\n' '{print "on",$0,"=",$0,"and"}'
on a.INFOCODE = a.INFOCODE and
on a.SOURCENAME = a.SOURCENAME and
on a.SOURCETYPE = a.SOURCETYPE and
on a.PUBLISHTYPE = a.PUBLISHTYPE and
on a.NOTICEDATE = a.NOTICEDATE and
on a.ENDDATE = a.ENDDATE and
on a.NOTICETITLE = a.NOTICETITLE and
on a.LANGUAGE = a.LANGUAGE and
on a.IMPORTLEVEL = a.IMPORTLEVEL and
on a.SOURCEURL = a.SOURCEURL and
on a.ATTACHTYPE = a.ATTACHTYPE and
on a.ATTACHNAME = a.ATTACHNAME and
on a.ATTACHSIZE = a.ATTACHSIZE and
on a.FORM = a.FORM and
on a.ACCESSORYNUM = a.ACCESSORYNUM and
on a.NOTICESTATE = a.NOTICESTATE and
on a.PUBLISHDATE = a.PUBLISHDATE and
on a.FILENUMBER = a.FILENUMBER and

替換
root@bd-dev-mingshuo-183:/tmp#more 1|gawk 'gsub(/,/,"\n",$0)'|gawk -F'\n' '{print $0,"=",$0,"and"}'|sed 's/= a/= b/g'     
a.INFOCODE = b.INFOCODE and
a.SOURCENAME = b.SOURCENAME and
a.SOURCETYPE = b.SOURCETYPE and
a.PUBLISHTYPE = b.PUBLISHTYPE and
a.NOTICEDATE = b.NOTICEDATE and
a.ENDDATE = b.ENDDATE and
a.NOTICETITLE = b.NOTICETITLE and
a.LANGUAGE = b.LANGUAGE and
a.IMPORTLEVEL = b.IMPORTLEVEL and
a.SOURCEURL = b.SOURCEURL and
a.ATTACHTYPE = b.ATTACHTYPE and
a.ATTACHNAME = b.ATTACHNAME and
a.ATTACHSIZE = b.ATTACHSIZE and
a.FORM = b.FORM and
a.ACCESSORYNUM = b.ACCESSORYNUM and
a.NOTICESTATE = b.NOTICESTATE and
a.PUBLISHDATE = b.PUBLISHDATE and
a.FILENUMBER = b.FILENUMBER and

處理過程比較簡單，重點在于gawk里的gsub函數(shù)的應(yīng)用，以及處理思路。

到此，相信大家對“gawk gsub函數(shù)的實際應(yīng)用”有了更深的了解，不妨來實際操作一番吧！這里是億速云網(wǎng)站，更多相關(guān)內(nèi)容可以進(jìn)入相關(guān)頻道進(jìn)行查詢，關(guān)注我們，繼續(xù)學(xué)習(xí)！