您好,登錄后才能下訂單哦!
最近在自學(xué)爬蟲框架scrapy,看了看官網(wǎng)文檔,迫不及待動(dòng)手安裝。結(jié)果錯(cuò)誤百出,慘不忍睹。網(wǎng)上搜了一篇文章,(原文鏈接 http://www.tuicool.com/articles/URNVV3E)。
首先,介紹下環(huán)境:
1、Centos6.5 x64 安裝在VMware虛擬機(jī)中,因?yàn)楹笃谛枰浦?,遂選擇了兼容vm10.0。
# yum -y update
升級(jí)下系統(tǒng)
2、Python2.7.9。系統(tǒng)自帶為2.6.6。
cd ~/Download
管網(wǎng)下載源碼:
wget --no-check-certificate https://www.python.org/ftp/python/2.7.9/Python-2.7.9.tar.xz
tar xvf Python-2.7.9.tar.xz
cd Python-2.7.9
./configure --with-ensurepip=install #其余保持默認(rèn)
make
sudo make install # 如果python2.7.9做副版本的話使用make altinstall
[Mikky@localhost Python-2.7.9]$ python
Python 2.7.9 (default, Feb 1 2016, 21:30:54)
[GCC 4.4.7 20120313 (Red Hat 4.4.7-16)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>>
升級(jí)了下libffi-devel因?yàn)椴人犹?,所以直接又更新了下,此步選做:
sudo yum install update libffi-devel
至此環(huán)境準(zhǔn)備妥當(dāng),下面就是踩坑環(huán)節(jié)o(╯□╰)o
安裝失敗方法:
sudo /usr/local/bin/pip install scrapy
錯(cuò)誤很多,參考文首的鏈接依然無法解決??泳筒毁N出來了,慢慢踩吧。
那么多坑的主要原因在于pip安裝軟件的依賴庫問題,尤其是:cryptography。
pip默認(rèn)安裝最新的而不是最穩(wěn)定的版本,安裝的cryptography為最新版(1.2.2 我的安裝時(shí)間為2016年2月初),因此:
sudo /usr/local/bin/pip install cryptography==0.9
........無視warn
Successfully installed cryptography idna pyasn1 six enum34 ipaddress cffi pycparser
Cleaning up...
到這大功告成一大半了,接下來請(qǐng)主角出場(chǎng):
sudo /usr/local/bin/pip install scrapy
.........
Successfully installed scrapy pyOpenSSL queuelib service-identity lxml w3lib cssselect Twisted pyasn1-modules characteristic zope.interface
Cleaning up...
[Mikky@localhost ~]$ scrapy version
/usr/local/lib/python2.7/site-packages/cffi/model.py:526: UserWarning: 'point_conversion_form_t' has no values explicitly defined; next version will refuse to guess which integer type it is meant to be (unsigned/signed, int/long)
% self._get_c_name())
Scrapy 1.0.4
繼續(xù)無視警告,已經(jīng)被這些警告整瘋了。
$ scrapy startproject tutorial
/usr/local/lib/python2.7/site-packages/cffi/model.py:526: UserWarning: 'point_conversion_form_t' has no values explicitly defined; next version will refuse to guess which integer type it is meant to be (unsigned/signed, int/long)
% self._get_c_name())
New Scrapy project 'tutorial' created in:
/home/Mikky/scrapy/tutorial
You can start your first spider with:
cd tutorial
scrapy genspider example example.com
[Mikky@localhost scrapy]$ ll
total 4
drwxrwxr-x. 3 Mikky Mikky 4096 Feb 3 22:27 tutorial
安裝完成,關(guān)于這個(gè)warning,參見https://bitbucket.org/cffi/cffi/issues/234/cffi-131-userwarning-about 。
免責(zé)聲明:本站發(fā)布的內(nèi)容(圖片、視頻和文字)以原創(chuàng)、轉(zhuǎn)載和分享為主,文章觀點(diǎn)不代表本網(wǎng)站立場(chǎng),如果涉及侵權(quán)請(qǐng)聯(lián)系站長(zhǎng)郵箱:is@yisu.com進(jìn)行舉報(bào),并提供相關(guān)證據(jù),一經(jīng)查實(shí),將立刻刪除涉嫌侵權(quán)內(nèi)容。