SkLearning管道匹配:属性错误:找不到下部

本教程将介绍SkLearning管道匹配:属性错误:找不到下部的处理方法,这篇教程是从别的地方看到的,然后加了一些国外程序员的疑问与解答,希望能对你有所帮助,好了,下面开始学习吧。

SkLearning管道匹配:属性错误:找不到下部 教程 第1张

问题描述

我想在skLearning中使用管道,如下所示:

corpus = load_files('corpus/train')

stop_words = [x for x in open('stopwords.txt', 'r').read().split('
')]  # Uppercase!

countvec = CountVectorizer(stop_words=stop_words, ngram_range=(1, 2))

X_train, X_test, y_train, y_test = train_test_split(corpus.data, corpus.target, test_size=0.9,
 random_state=0)
x_train_counts = countvec.fit_transform(X_train)
x_test_counts = countvec.transform(X_test)

k_fold = KFold(n=len(corpus.data), n_folds=6)
confusion = np.array([[0, 0], [0, 0]])

pipeline = Pipeline([
 ('vectorizer',  CountVectorizer(stop_words=stop_words, ngram_range=(1, 2))),
 ('classifier',  MultinomialNB()) ])

for train_indices, test_indices in k_fold:

 pipeline.fit(x_train_counts, y_train)
 predictions = pipeline.predict(x_test_counts)

但是,我收到此错误:

AttributeError: lower not found

我看过这个帖子:

AttributeError: lower not found; using a Pipeline with a CountVectorizer in scikit-learn

但我将字节列表传递给向量器,所以这不应该是问题所在。

编辑

corpus = load_files('corpus')

stop_words = [x for x in open('stopwords.txt', 'r').read().split('
')]

X_train, X_test, y_train, y_test = train_test_split(corpus.data, corpus.target, test_size=0.5,
 random_state=0)

k_fold = KFold(n=len(corpus.data), n_folds=6)
confusion = np.array([[0, 0], [0, 0]])

pipeline = Pipeline([
 ('vectorizer', CountVectorizer(stop_words=stop_words, ngram_range=(1, 2))),
 ('classifier', MultinomialNB())])

for train_indices, test_indices in k_fold:
 pipeline.fit(X_train[train_indices], y_train[train_indices])
 predictions = pipeline.predict(X_test[test_indices])

现在我收到错误:

TypeError: only integer arrays with one element can be converted to an index

第二次编辑

corpus = load_files('corpus')

stop_words = [y for x in open('stopwords.txt', 'r').read().split('
') for y in (x, x.title())]

k_fold = KFold(n=len(corpus.data), n_folds=6)
confusion = np.array([[0, 0], [0, 0]])

pipeline = Pipeline([
 ('vectorizer', CountVectorizer(stop_words=stop_words, ngram_range=(1, 2))),
 ('classifier', MultinomialNB())])

for train_indices, test_indices in k_fold:
 pipeline.fit(corpus.data, corpus.target)

推荐答案

您没有正确使用管道。您不需要将数据矢量化,其想法是管道将数据矢量化。

# This is done by the pipeline
# x_train_counts = countvec.fit_transform(X_train)
# x_test_counts = countvec.transform(X_test)

k_fold = KFold(n=len(corpus.data), n_folds=6)
confusion = np.array([[0, 0], [0, 0]])

pipeline = Pipeline([
 ('vectorizer',  CountVectorizer(stop_words=stop_words, ngram_range=(1, 2))),
 ('classifier',  MultinomialNB()) ])

# also you are not using the indices...
for train_indices, test_indices in k_fold:

 pipeline.fit(corpus.data[train_indices], corpus.target[train_indices])
 predictions = pipeline.predict(corpus.data[test_indices])

好了关于SkLearning管道匹配:属性错误:找不到下部的教程就到这里就结束了,希望趣模板源码网找到的这篇技术文章能帮助到大家,更多技术教程可以在站内搜索。