国产精品天干天干,亚洲毛片在线,日韩gay小鲜肉啪啪18禁,女同Gay自慰喷水

歡迎光臨散文網(wǎng) 會員登陸 & 注冊

百度LAC分詞(python3實現(xiàn))

2023-02-24 17:32 作者:trisyp  | 我要投稿

對應(yīng)數(shù)據(jù)的鏈接放這里(https://download.csdn.net/download/Trisyp/87486282)了

import pandas as pd
from util.logger import Log
import os
from util.data_dir import root_dir
from LAC import LAC
os_file_name = os.path.split(os.path.realpath(__file__))[-1]


# 加載LAC模型
lac = LAC(mode="lac")
# 載入自定義詞典
lac.load_customization(f'{root_dir}/data/lac_data/lac_define_dict.txt')


def load_stopwords():
? ?"""
? ?加載停用詞
? ?:return: list
? ?"""
? ?stopwords_path = f'{root_dir}/data/lac_data/stop_words.txt'
? ?stopwords_increase_path = f'{root_dir}/data/lac_data/stop_words_increase.txt'
? ?if stopwords_path:
? ? ? ?with open(stopwords_path, 'r', encoding='utf-8-sig') as f:
? ? ? ? ? ?stopwords = [line.strip() for line in f]
? ?else:
? ? ? ?Log().info(os_file_name, '《熱點事件》|沒有停用詞表|')
? ? ? ?stopwords = []
? ?if stopwords_increase_path:
? ? ? ?with open(stopwords_increase_path, 'r', encoding='utf-8-sig') as f:
? ? ? ? ? ?stopwords_increase = [line.strip() for line in f]
? ? ? ?stopwords.extend(stopwords_increase)
? ?else:
? ? ? ?Log().info(os_file_name, '《熱點事件》|沒有停用詞增加表|')
? ?return stopwords


def extract_important_word(content: str):
? ?"""
? ?LAC分詞
? ?:param content: str
? ?:return: str
? ?"""

? ?# 運行l(wèi)ac
? ?lac_result = lac.run(content)

? ?# 將分詞和詞性轉(zhuǎn)為dataframe
? ?lac_result_df = pd.DataFrame(data=zip(lac_result[1], lac_result[0]), columns=['char', 'word'])

? ?# 選取重要詞性的dataframe
? ?seg_data = list(lac_result_df[(lac_result_df['char'] == 'n') | (lac_result_df['char'] == 'vn')
? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?| (lac_result_df['char'] == 'an') | (lac_result_df['char'] == 'nz')
? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?| (lac_result_df['char'] == 'ORG') | (lac_result_df['char'] == 'v')
? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?| (lac_result_df['char'] == 'LOC')].word)

? ?# 獲取重要詞語,不在停用詞內(nèi)
? ?docu_process = ','.join([word for word in seg_data if len(word) >= 2 and word not in load_stopwords()])
? ?return docu_process


if __name__ == "__main__":
? ?sentence = "小區(qū)物業(yè)沒人管,垃圾也沒人處理"
? ?sentence = extract_important_word(sentence)
? ?print(sentence)

百度LAC分詞(python3實現(xiàn))的評論 (共 條)

分享到微博請遵守國家法律
仪陇县| 紫阳县| 丽水市| 榕江县| 衡东县| 咸丰县| 开封县| 博野县| 玛曲县| 吴旗县| 漠河县| 荃湾区| 宣恩县| 香河县| 合阳县| 津南区| 长沙县| 汤阴县| 平罗县| 无棣县| 西宁市| 湖北省| 白朗县| 泾川县| 壶关县| 杭州市| 郸城县| 井研县| 青河县| 广元市| 乌拉特前旗| 旬阳县| 富源县| 百色市| 黔江区| 新巴尔虎右旗| 邵东县| 海南省| 榆中县| 凤翔县| 张家港市|