您好,登錄后才能下訂單哦!
本文小編為大家詳細(xì)介紹“Python怎么使用tf-idf算法計(jì)算文檔關(guān)鍵字權(quán)重并生成詞云”,內(nèi)容詳細(xì),步驟清晰,細(xì)節(jié)處理妥當(dāng),希望這篇“Python怎么使用tf-idf算法計(jì)算文檔關(guān)鍵字權(quán)重并生成詞云”文章能幫助大家解決疑惑,下面跟著小編的思路慢慢深入,一起來(lái)學(xué)習(xí)新知識(shí)吧。
代碼如下:
注意需要安裝pip install sklean
;
from re import split from jieba.posseg import dt from sklearn.feature_extraction.text import TfidfVectorizer from collections import Counter from time import time import jieba #pip install sklean FLAGS = set('a an b f i j l n nr nrfg nrt ns nt nz s t v vi vn z eng'.split()) def cut(text): for sentence in split('[^a-zA-Z0-9\u4e00-\u9fa5]+', text.strip()): for w in dt.cut(sentence): if len(w.word) > 2 and w.flag in FLAGS: yield w.word class TFIDF: def __init__(self, idf): self.idf = idf @classmethod def train(cls, texts): model = TfidfVectorizer(tokenizer=cut) model.fit(texts) idf = {w: model.idf_[i] for w, i in model.vocabulary_.items()} return cls(idf) def get_idf(self, word): return self.idf.get(word, max(self.idf.values())) def extract(self, text, top_n=10): counter = Counter() for w in cut(text): counter[w] += self.get_idf(w) #return [i[0:2] for i in counter.most_common(top_n)] return [i[0] for i in counter.most_common(top_n)] if __name__ == '__main__': t0 = time() with open('./nlp-homework.txt', encoding='utf-8')as f: _texts = f.read().strip().split('\n') # print(_texts) tfidf = TFIDF.train(_texts) # print(_texts) for _text in _texts: seq_list=jieba.cut(_text,cut_all=True) #全模式 # seq_list=jieba.cut(_text,cut_all=False) #精確模式 # seq_list=jieba.cut_for_search(_text,) #搜索引擎模式 # print(list(seq_list)) print(tfidf.extract(_text)) with open('./resultciyun.txt','a+', encoding='utf-8') as g: for i in tfidf.extract(_text): g.write(str(i) + " ") print(time() - t0)
代碼如下:
注意需要安裝pip install wordcloud
;
以及為了保證中文字體正常顯示,需要下載SimSun.ttf
字體,并且將這個(gè)字體包也放在和程序相同的目錄下;
from wordcloud import WordCloud filename = "resultciyun.txt" with open(filename) as f: resultciyun = f.read() wordcloud = WordCloud(font_path="simsun.ttf").generate(resultciyun) # %pylab inline import matplotlib.pyplot as plt plt.imshow(wordcloud, interpolation='bilinear') plt.axis("off") plt.show()
讀到這里,這篇“Python怎么使用tf-idf算法計(jì)算文檔關(guān)鍵字權(quán)重并生成詞云”文章已經(jīng)介紹完畢,想要掌握這篇文章的知識(shí)點(diǎn)還需要大家自己動(dòng)手實(shí)踐使用過(guò)才能領(lǐng)會(huì),如果想了解更多相關(guān)內(nèi)容的文章,歡迎關(guān)注億速云行業(yè)資訊頻道。
免責(zé)聲明:本站發(fā)布的內(nèi)容(圖片、視頻和文字)以原創(chuàng)、轉(zhuǎn)載和分享為主,文章觀點(diǎn)不代表本網(wǎng)站立場(chǎng),如果涉及侵權(quán)請(qǐng)聯(lián)系站長(zhǎng)郵箱:is@yisu.com進(jìn)行舉報(bào),并提供相關(guān)證據(jù),一經(jīng)查實(shí),將立刻刪除涉嫌侵權(quán)內(nèi)容。