您好,登錄后才能下訂單哦!
這篇文章主要講解了python中如何使用Beautiful Soup,內(nèi)容清晰明了,相信大家閱讀完之后會有幫助。
Beautiful Soup就是Python的一個(gè)HTML或XML的解析庫,可以用它來方便地從網(wǎng)頁中提取數(shù)據(jù)。它有如下三個(gè)特點(diǎn):
首先,我們要安裝它:pip install bs4,然后安裝 pip install beautifulsoup4.
Beautiful Soup支持的解析器
下面我們以lxml解析器為例:
from bs4 import BeautifulSoup
soup = BeautifulSoup('<p>Hello</p>', 'lxml')
print(soup.p.string)
結(jié)果:
Hello
beautiful soup美化的效果實(shí)例:
html = """ <html><head><title>The Dormouse's story</title></head> <body> <p class="title" name="dromouse"><b>The Dormouse's story</b></p> <p class="story">Once upon a time there were three little sisters; and their names were <a href="http://example.com/elsie" rel="external nofollow" rel="external nofollow" rel="external nofollow" class="sister" id="link1"><!-- Elsie --></a>, <a href="http://example.com/lacie" rel="external nofollow" rel="external nofollow" rel="external nofollow" class="sister" id="link2">Lacie</a> and <a href="http://example.com/tillie" rel="external nofollow" rel="external nofollow" rel="external nofollow" class="sister" id="link3">Tillie</a>; and they lived at the bottom of a well.</p> <p class="story">...</p> """ from bs4 import BeautifulSoup soup = BeautifulSoup(html, 'lxml')#調(diào)用prettify()方法。這個(gè)方法可以把要解析的字符串以標(biāo)準(zhǔn)的縮進(jìn)格式輸出 print(soup.prettify()) print(soup.title.string)
結(jié)果:
<html> <head> <title> The Dormouse's story </title> </head> <body> <p class="title" name="dromouse"> <b> The Dormouse's story </b> </p> <p class="story"> Once upon a time there were three little sisters; and their names were <a class="sister" href="http://example.com/elsie" rel="external nofollow" rel="external nofollow" rel="external nofollow" id="link1"> <!-- Elsie --> </a> , <a class="sister" href="http://example.com/lacie" rel="external nofollow" rel="external nofollow" rel="external nofollow" id="link2"> Lacie </a> and <a class="sister" href="http://example.com/tillie" rel="external nofollow" rel="external nofollow" rel="external nofollow" id="link3"> Tillie </a> ; and they lived at the bottom of a well. </p> <p class="story"> ... </p> </body> </html> The Dormouse's story
下面舉例說明選擇元素、屬性、名稱的方法
html = """ <html><head><title>The Dormouse's story</title></head> <body> <p class="title" name="dromouse"><b>The Dormouse's story</b></p> <p class="story">Once upon a time there were three little sisters; and their names were <a href="http://example.com/elsie" rel="external nofollow" rel="external nofollow" rel="external nofollow" class="sister" id="link1"><!-- Elsie --></a>, <a href="http://example.com/lacie" rel="external nofollow" rel="external nofollow" rel="external nofollow" class="sister" id="link2">Lacie</a> and <a href="http://example.com/tillie" rel="external nofollow" rel="external nofollow" rel="external nofollow" class="sister" id="link3">Tillie</a>; and they lived at the bottom of a well.</p> <p class="story">...</p> """ from bs4 import BeautifulSoup soup = BeautifulSoup(html, 'lxml') print('輸出結(jié)果為title節(jié)點(diǎn)加里面的文字內(nèi)容:\n',soup.title) print('輸出它的類型:\n',type(soup.title)) print('輸出節(jié)點(diǎn)的文本內(nèi)容:\n',soup.title.string) print('結(jié)果是節(jié)點(diǎn)加其內(nèi)部的所有內(nèi)容:\n',soup.head) print('結(jié)果是第一個(gè)p節(jié)點(diǎn)的內(nèi)容:\n',soup.p) print('利用name屬性獲取節(jié)點(diǎn)的名稱:\n',soup.title.name) #這里需要注意的是,有的返回結(jié)果是字符串,有的返回結(jié)果是字符串組成的列表。 # 比如,name屬性的值是唯一的,返回的結(jié)果就是單個(gè)字符串。 # 而對于class,一個(gè)節(jié)點(diǎn)元素可能有多個(gè)class,所以返回的是列表。 print('每個(gè)節(jié)點(diǎn)可能有多個(gè)屬性,比如id和class等:\n',soup.p.attrs) print('選擇這個(gè)節(jié)點(diǎn)元素后,可以調(diào)用attrs獲取所有屬性:\n',soup.p.attrs['name']) print('獲取p標(biāo)簽的name屬性值:\n',soup.p['name']) print('獲取p標(biāo)簽的class屬性值:\n',soup.p['class']) print('獲取第一個(gè)p節(jié)點(diǎn)的文本:\n',soup.p.string)
結(jié)果:
輸出結(jié)果為title節(jié)點(diǎn)加里面的文字內(nèi)容: <title>The Dormouse's story</title> 輸出它的類型: <class 'bs4.element.Tag'> 輸出節(jié)點(diǎn)的文本內(nèi)容: The Dormouse's story 結(jié)果是節(jié)點(diǎn)加其內(nèi)部的所有內(nèi)容: <head><title>The Dormouse's story</title></head> 結(jié)果是第一個(gè)p節(jié)點(diǎn)的內(nèi)容: <p class="title" name="dromouse"><b>The Dormouse's story</b></p> 利用name屬性獲取節(jié)點(diǎn)的名稱: title 每個(gè)節(jié)點(diǎn)可能有多個(gè)屬性,比如id和class等: {'class': ['title'], 'name': 'dromouse'} 選擇這個(gè)節(jié)點(diǎn)元素后,可以調(diào)用attrs獲取所有屬性: dromouse 獲取p標(biāo)簽的name屬性值: dromouse 獲取p標(biāo)簽的class屬性值: ['title'] 獲取第一個(gè)p節(jié)點(diǎn)的文本: The Dormouse's story
在上面的例子中,我們知道每一個(gè)返回結(jié)果都是bs4.element.Tag類型,它同樣可以繼續(xù)調(diào)用節(jié)點(diǎn)進(jìn)行下一步的選擇。
html = """ <html><head><title>The Dormouse's story</title></head> <body> """ from bs4 import BeautifulSoup soup = BeautifulSoup(html, 'lxml') print('獲取了head節(jié)點(diǎn)元素,繼續(xù)調(diào)用head來選取其內(nèi)部的head節(jié)點(diǎn)元素:\n',soup.head.title) print('繼續(xù)調(diào)用輸出類型:\n',type(soup.head.title)) print('繼續(xù)調(diào)用輸出內(nèi)容:\n',soup.head.title.string)
結(jié)果:
獲取了head節(jié)點(diǎn)元素,繼續(xù)調(diào)用head來選取其內(nèi)部的head節(jié)點(diǎn)元素: <title>The Dormouse's story</title> 繼續(xù)調(diào)用輸出類型: <class 'bs4.element.Tag'> 繼續(xù)調(diào)用輸出內(nèi)容: The Dormouse's story
(1)find_all()
find_all,顧名思義,就是查詢所有符合條件的元素。給它傳入一些屬性或文本,就可以得到符合條件的元素,它的功能十分強(qiáng)大。
find_all(name , attrs , recursive , text , **kwargs)
他的用法:
html=''' <div class="panel"> <div class="panel-heading"> <h5>Hello</h5> </div> <div class="panel-body"> <ul class="list" id="list-1"> <li class="element">Foo</li> <li class="element">Bar</li> <li class="element">Jay</li> </ul> <ul class="list list-small" id="list-2"> <li class="element">Foo</li> <li class="element">Bar</li> </ul> </div> </div> ''' from bs4 import BeautifulSoup soup = BeautifulSoup(html, 'lxml') print('查詢所有ul節(jié)點(diǎn),返回結(jié)果是列表類型,長度為2:\n',soup.find_all(name='ul')) print('每個(gè)元素依然都是bs4.element.Tag類型:\n',type(soup.find_all(name='ul')[0])) #將以上步驟換一種方式,遍歷出來 for ul in soup.find_all(name='ul'): print('輸出每個(gè)u1:',ul.find_all(name='li')) #遍歷兩層 for ul in soup.find_all(name='ul'): print('輸出每個(gè)u1:',ul.find_all(name='li')) for li in ul.find_all(name='li'): print('輸出每個(gè)元素:',li.string)
結(jié)果:
查詢所有ul節(jié)點(diǎn),返回結(jié)果是列表類型,長度為2: [<ul class="list" id="list-1"> <li class="element">Foo</li> <li class="element">Bar</li> <li class="element">Jay</li> </ul>, <ul class="list list-small" id="list-2"> <li class="element">Foo</li> <li class="element">Bar</li> </ul>] 每個(gè)元素依然都是bs4.element.Tag類型: <class 'bs4.element.Tag'> 輸出每個(gè)u1: [<li class="element">Foo</li>, <li class="element">Bar</li>, <li class="element">Jay</li>] 輸出每個(gè)u1: [<li class="element">Foo</li>, <li class="element">Bar</li>] 輸出每個(gè)u1: [<li class="element">Foo</li>, <li class="element">Bar</li>, <li class="element">Jay</li>] 輸出每個(gè)元素: Foo 輸出每個(gè)元素: Bar 輸出每個(gè)元素: Jay 輸出每個(gè)u1: [<li class="element">Foo</li>, <li class="element">Bar</li>] 輸出每個(gè)元素: Foo 輸出每個(gè)元素: Bar
看完上述內(nèi)容,有沒有對python中如何使用Beautiful Soup有進(jìn)一步的了解,如果還想學(xué)習(xí)更多內(nèi)容,歡迎關(guān)注億速云行業(yè)資訊頻道。
免責(zé)聲明:本站發(fā)布的內(nèi)容(圖片、視頻和文字)以原創(chuàng)、轉(zhuǎn)載和分享為主,文章觀點(diǎn)不代表本網(wǎng)站立場,如果涉及侵權(quán)請聯(lián)系站長郵箱:is@yisu.com進(jìn)行舉報(bào),并提供相關(guān)證據(jù),一經(jīng)查實(shí),將立刻刪除涉嫌侵權(quán)內(nèi)容。