NLP（新闻文本分类）——数据读取与数据分析.pdfVIP

NLP （新闻⽂本分类）——数据读取与数据分析 初始数据 import pandas as pd df_train = pd .read_csv(E:/python-project/deep-learning/datawhale/nlp/news-data/train_set.csv/train_set.csv, sep=\t) df_test = pd .read_csv(E:/python-project/deep-learning/datawhale/nlp/news-data/test_a.csv/test_a.csv, sep=\t) train test 长度分布 import numpy as np import matplotlib.pyplot as plt def split_df(df_row): return len(str(df_row).split()) len_dist = np .vectorize(split_df)(df_train[text]) plt.figure() plt.hist(x=len_dist, bins=100) plt.xlim([0,max(len_dist)]) plt.xlabel(length of samp