您好,登錄后才能下訂單哦!
這篇文章主要介紹python如何實(shí)現(xiàn)textrank關(guān)鍵詞提取,文中介紹的非常詳細(xì),具有一定的參考價(jià)值,感興趣的小伙伴們一定要看完!
用python寫了一個(gè)簡單版本的textrank,實(shí)現(xiàn)提取關(guān)鍵詞的功能。
import numpy as np import jieba import jieba.posseg as pseg class TextRank(object): def __init__(self, sentence, window, alpha, iternum): self.sentence = sentence self.window = window self.alpha = alpha self.edge_dict = {} #記錄節(jié)點(diǎn)的邊連接字典 self.iternum = iternum#迭代次數(shù) #對句子進(jìn)行分詞 def cutSentence(self): jieba.load_userdict('user_dict.txt') tag_filter = ['a','d','n','v'] seg_result = pseg.cut(self.sentence) self.word_list = [s.word for s in seg_result if s.flag in tag_filter] print(self.word_list) #根據(jù)窗口,構(gòu)建每個(gè)節(jié)點(diǎn)的相鄰節(jié)點(diǎn),返回邊的集合 def createNodes(self): tmp_list = [] word_list_len = len(self.word_list) for index, word in enumerate(self.word_list): if word not in self.edge_dict.keys(): tmp_list.append(word) tmp_set = set() left = index - self.window + 1#窗口左邊界 right = index + self.window#窗口右邊界 if left < 0: left = 0 if right >= word_list_len: right = word_list_len for i in range(left, right): if i == index: continue tmp_set.add(self.word_list[i]) self.edge_dict[word] = tmp_set #根據(jù)邊的相連關(guān)系,構(gòu)建矩陣 def createMatrix(self): self.matrix = np.zeros([len(set(self.word_list)), len(set(self.word_list))]) self.word_index = {}#記錄詞的index self.index_dict = {}#記錄節(jié)點(diǎn)index對應(yīng)的詞 for i, v in enumerate(set(self.word_list)): self.word_index[v] = i self.index_dict[i] = v for key in self.edge_dict.keys(): for w in self.edge_dict[key]: self.matrix[self.word_index[key]][self.word_index[w]] = 1 self.matrix[self.word_index[w]][self.word_index[key]] = 1 #歸一化 for j in range(self.matrix.shape[1]): sum = 0 for i in range(self.matrix.shape[0]): sum += self.matrix[i][j] for i in range(self.matrix.shape[0]): self.matrix[i][j] /= sum #根據(jù)textrank公式計(jì)算權(quán)重 def calPR(self): self.PR = np.ones([len(set(self.word_list)), 1]) for i in range(self.iternum): self.PR = (1 - self.alpha) + self.alpha * np.dot(self.matrix, self.PR) #輸出詞和相應(yīng)的權(quán)重 def printResult(self): word_pr = {} for i in range(len(self.PR)): word_pr[self.index_dict[i]] = self.PR[i][0] res = sorted(word_pr.items(), key = lambda x : x[1], reverse=True) print(res) if __name__ == '__main__': s = '程序員(英文Programmer)是從事程序開發(fā)、維護(hù)的專業(yè)人員。一般將程序員分為程序設(shè)計(jì)人員和程序編碼人員,但兩者的界限并不非常清楚,特別是在中國。軟件從業(yè)人員分為初級程序員、高級程序員、系統(tǒng)分析員和項(xiàng)目經(jīng)理四大類。' tr = TextRank(s, 3, 0.85, 700) tr.cutSentence() tr.createNodes() tr.createMatrix() tr.calPR() tr.printResult()
以上是“python如何實(shí)現(xiàn)textrank關(guān)鍵詞提取”這篇文章的所有內(nèi)容,感謝各位的閱讀!希望分享的內(nèi)容對大家有幫助,更多相關(guān)知識,歡迎關(guān)注億速云行業(yè)資訊頻道!
免責(zé)聲明:本站發(fā)布的內(nèi)容(圖片、視頻和文字)以原創(chuàng)、轉(zhuǎn)載和分享為主,文章觀點(diǎn)不代表本網(wǎng)站立場,如果涉及侵權(quán)請聯(lián)系站長郵箱:is@yisu.com進(jìn)行舉報(bào),并提供相關(guān)證據(jù),一經(jīng)查實(shí),將立刻刪除涉嫌侵權(quán)內(nèi)容。