您好,登錄后才能下訂單哦!
這篇文章將為大家詳細(xì)講解有關(guān)python如何爬取淘寶某關(guān)鍵詞的所有商品及其信息,小編覺得挺實(shí)用的,因此分享給大家做個(gè)參考,希望大家閱讀完這篇文章后可以有所收獲。
源代碼:https://github.com/Ckend/taobao_scrapy
源代碼包含兩個(gè)Python文件,get_items.py 用于通過關(guān)鍵詞獲得商品基本信息,包括商品名稱、店鋪、銷量、評(píng)論數(shù)、圖片、優(yōu)惠、出貨地、原價(jià)等等。輸出格式為.csv文件。
另一個(gè)Python文件: get_review_cloud.py 用于通過userID和itemID (從get_items.py中獲得這兩個(gè)信息) 獲得某商品的評(píng)論云。輸出.csv文件。
源代碼很簡(jiǎn)單,十幾行,邏輯如下:
1. 訪問接口(f=urllib.request.urlopen(url))
2. 讀取源代碼并解碼為utf-8然后轉(zhuǎn)化為json
3. 寫入至.csv文件。
以下給出get_items.py的內(nèi)容
get_items.py:
import urllib.request
import urllib.parse
import json
import csv
def write_csv(data, filename):
count = 0
with open(filename, 'a') as outf:
dw = csv.DictWriter(outf, fieldnames=["title","sold","commentCount","item_id","shipping","fastPostFee","userId","nick","userType","isB2c","location","sellerLoc","pic_path","type","tItemType","zkType","zkGroup","priceColor","priceWithRate","auctionURL","isP4p","itemNumId","originalPrice","freight","act","coinLimit","priceWap","price","category","auctionType","url","img2","wwimUrl","previewUrl","favoriteUrl","isMobileEcard","iswebp","name","iconList","icons","area"])
if count == 0:
# 第一行才寫入頭部
dw.writeheader()
count += 1
for row in data:
dw.writerow(row)
def get_items(searchWords, page):
url = "https://s.m.taobao.com/search?event_submit_do_new_search_auction=1&_input_charset=utf-8&topSearch=1&atype=b&searchfrom=1&action=home%3Aredirect_app_action&from=1&q="+ str(urllib.parse.quote(searchWords)) +"&sst=1&n=20&buying=buyitnow&m=api4h6&token4h6=&abtest=20&wlsort=20&page="+str(page)
f = urllib.request.urlopen(url)
result = json.loads(f.read().decode('utf-8'))
return result['listItem']
def get_items_by_keywords(searchWords, page):
for i in range(1,page):
result = get_items(searchWords ,i)
try:
write_csv(result,"./result/"+str(searchWords)+"_result.csv")
except INdexError:
# 關(guān)鍵詞產(chǎn)品已抓取完
print("該關(guān)鍵詞產(chǎn)品已全部抓取完畢")
get_items_by_keywords("空氣清新器", 100)
關(guān)于“python如何爬取淘寶某關(guān)鍵詞的所有商品及其信息”這篇文章就分享到這里了,希望以上內(nèi)容可以對(duì)大家有一定的幫助,使各位可以學(xué)到更多知識(shí),如果覺得文章不錯(cuò),請(qǐng)把它分享出去讓更多的人看到。
免責(zé)聲明:本站發(fā)布的內(nèi)容(圖片、視頻和文字)以原創(chuàng)、轉(zhuǎn)載和分享為主,文章觀點(diǎn)不代表本網(wǎng)站立場(chǎng),如果涉及侵權(quán)請(qǐng)聯(lián)系站長(zhǎng)郵箱:is@yisu.com進(jìn)行舉報(bào),并提供相關(guān)證據(jù),一經(jīng)查實(shí),將立刻刪除涉嫌侵權(quán)內(nèi)容。