LDA - 文档主题生成模型

LDA（Latent Dirichlet Allocation）是一种文档主题生成模型，也称为一个三层贝叶斯概率模型，包含词、主题和文档三层结构。

所谓生成模型，就是说，一篇文章的每个词都是通过 “以一定概率选择了某个主题，并从这个主题中以一定概率选择某个词语” 这样一个过程得到，先从文档选择好主题，然后从主题里选择词。

文档到主题服从多项式分布，主题到词服从多项式分布。

LDA 生成：

• 1. 确定一个文档中的单词数。假设我们的文档有六个单词。
• 2. 确定该文档由哪些主题混合而来，例如，这个文档包含 1/2 的“健康”（health）主题和 1/2 的“蔬菜”（vegetables）主题。
• 3. 用每个主题的多项分布生成的单词来填充文档中的单词槽。在我们的例子中，“健康”主题占文档的 1/2，或者说占三个词。“健康”主题有“diet”这个词的可能性是 20%，或者有“execise" 这个词的概率是 15%，单词槽就是基于这些概率来填充的。

示例：Python 的 gensim 包中的 ldamodel

from nltk.tokenize import RegexpTokenizer
from stop_words import get_stop_words
from nltk.stem.porter import PorterStemmer
from gensim import corpora, models
import gensim

tokenizer = RegexpTokenizer(r'\w+')

# create English stop words list
en_stop = get_stop_words('en')

# Create p_stemmer of class PorterStemmer
p_stemmer = PorterStemmer()
    
# create sample documents
doc_a = "Brocolli is good to eat. My brother likes to eat good brocolli, but not my mother."
doc_b = "My mother spends a lot of time driving my brother around to baseball practice."
doc_c = "Some health experts suggest that driving may cause increased tension and blood pressure."
doc_d = "I often feel pressure to perform well at school, but my mother never seems to drive my brother to do better."
doc_e = "Health professionals say that brocolli is good for your health." 

# compile sample documents into a list
doc_set = [doc_a, doc_b, doc_c, doc_d, doc_e]

# list for tokenized documents in loop
texts = []

# loop through document list
for i in doc_set:
    
    # clean and tokenize document string
    raw = i.lower()
    tokens = tokenizer.tokenize(raw)

    # remove stop words from tokens
    stopped_tokens = [i for i in tokens if not i in en_stop]
    
    # stem tokens
    stemmed_tokens = [p_stemmer.stem(i) for i in stopped_tokens]
    
    # add tokens to list
    texts.append(stemmed_tokens)

# turn our tokenized documents into a id <-> term dictionary
dictionary = corpora.Dictionary(texts)
    
# convert tokenized documents into a document-term matrix
corpus = [dictionary.doc2bow(text) for text in texts]

# generate LDA model
ldamodel = gensim.models.ldamodel.LdaModel(corpus, num_topics=2, id2word = dictionary, passes=20)​