On Estimation and Selection for Topic Models

This article describes posterior maximization for topic models, identifying computational and conceptual gains from inference under a non-standard parametrization. We then show that fitted parameters can be used as the basis for a novel approach to marginal likelihood estimation, via block-diagonal approximation to the information matrix,that facilitates choosing the number of latent topics. This likelihood-based model selection is complemented with a goodness-of-fit analysis built around estimated residual dispersion. Examples are provided to illustrate model selection as well as to compare our estimation against standard alternative techniques.

• Publication year

  2011

• Venue

  International Conference on Artificial Intelligence and Statistics

• Publication date

  2011-09-21

• Fields of study

  Mathematics, Computer Science

• Identifiers
  arXiv 1109.4518
• External record

  Open on Semantic Scholar

• Source metadata

  Semantic Scholar

• No claims are published for this paper.

• No concepts are published for this paper.

• topicmodels: An R Package for Fitting Topic Models

  2011influential reference
• Determinant Approximations

  2011cited by this paper
• Inverse Regression for Analysis of Sentiment in Text

  2010cited by this paper
• A Bayesian Hierarchical Topic Model for Political Texts: Measuring Expressed Agendas in Senate Press Releases

  2010influential reference
• Reconceptualizing the classification of PNAS articles

  2010cited by this paper
• Latent Dirichlet Allocation

  2009cited by this paper
• On Smoothing and Inference for Topic Models

  2009cited by this paper
• Rethinking LDA: Why Priors Matter

  2009cited by this paper
• Efficient methods for topic model inference on streaming document collections

  2009cited by this paper
• Fast model-based estimation of ancestry in unrelated individuals.

  2009cited by this paper
• Graphical Models, Exponential Families, and Variational Inference

  2008cited by this paper
• Latent semantic analysis

  2008cited by this paper
• Supervised Topic Models

  2007cited by this paper
• Collapsed Variational Inference for HDP

  2007cited by this paper
• A correlated topic model of Science

  2007cited by this paper
• What Drives Media Slant? Evidence from U.S. Daily Newspapers

  2006cited by this paper
• Reconstructing genetic ancestry blocks in admixed individuals.

  2006influential reference
• Hierarchical Dirichlet Processes

  2006cited by this paper
• Finding scientific topics

  2004cited by this paper
• The Elements of Statistical Learning

  2003cited by this paper
• Inference of population structure using multilocus genotype data.

  2000cited by this paper
• Markov Chain Sampling Methods for Dirichlet Process Mixture Models

  2000cited by this paper
• Probabilistic latent semantic indexing

  1999cited by this paper
• Integrated likelihood methods for eliminating nuisance parameters

  1999cited by this paper
• Numerical analysis for statisticians

  1999cited by this paper
• Choice of Basis for Laplace Approximation

  1998cited by this paper
• Practical Bayesian Density Estimation Using Mixtures of Normals

  1997cited by this paper
• Computing Bayes Factors by Combining Simulation and Asymptotic Approximations

  1997cited by this paper
• Bayes factors

  1995cited by this paper
• Accurate Approximations for Posterior Moments and Marginal Densities

  1986cited by this paper
• Factor Analysis for Categorical Data

  1980cited by this paper
• The Analysis of Residuals in Cross-Classified Tables

  1973cited by this paper
• Estimating item parameters and latent ability when responses are scored in two or more nominal categories

  1972cited by this paper
• THIS IS A DRAFT VERSION. FINAL VERSION TO BE PUBLISHED AT NIPS ’06 A Collapsed Variational Bayesian Inference Algorithm for Latent Dirichlet Allocation

  year unknowninfluential reference

• Cancer vlog community building for social support on YouTube: a social capital perspective

  2026cites this paper
• Governing innovation: How technology maturity shapes R&D contracts

  2026cites this paper
• Global cultural appropriation discourse on reddit: a topic modeling approach

  2025cites this paper
• Living well? The unintended consequences of highly popular commercial fitness apps through social listening using Machine‐Assisted Topic Analysis: Evidence from X

  2025cites this paper
• Emerging technologies in sharing economy: a review and research agenda

  2025cites this paper
• Do LGBTIQ+ issues matter for populist radical right? An analysis of Italian parties’ social media narratives

  2025cites this paper
• Support Services for Positive College Experiences of Students With Disabilities in Higher Education: A Structured Topic Modeling Approach

  2025cites this paper
• A structural topic modeling of communication research: insights from over a century of journals' abstracts

  2025cites this paper
• Unveiling Undergraduate Research: Employing Ecological Momentary Assessment to Characterize and Compare Undergraduate Research Experiences

  2025cites this paper
• Perceptions of Long-Acting Injectable Pre-Exposure Prophylaxis Among Men Who Have Sex With Men and Transgender Individuals in Europe Using Structural Text Modeling Technique: Cross-Sectional Study

  2025cites this paper
• Recommend or not: A comparative analysis of customer reviews to uncover factors influencing explicit online recommendation behavior in peer-to-peer accommodation

  2024cites this paper
• Reasons Behind Words: OPEC Narratives and the Oil Market

  2024cites this paper
• “We are a part of the problem they protest”: judges, representation and racial justice statements

  2024cites this paper
• Technical language processing for Prognostics and Health Management: applying text similarity and topic modeling to maintenance work orders

  2024cites this paper
• Framing of Russia-Ukraine War: Comparisons of Global News

  2024cites this paper
• Disasters and technological upgrading measured by changes in demand for ICT labour: estimating the impacts with text

  2024cites this paper
• Analyzing relationships between latent topics in autonomous vehicle crash narratives and crash severity using natural language processing techniques and explainable XGBoost.

  2024cites this paper
• Comparing Airbnb and traditional accommodation experiences using text-mining methods – the hedonic and utilitarian values framework

  2024cites this paper
• What makes econometric ideas popular: The role of connectivity

  2024cites this paper
• A macroscope of English print culture, 1530–1700, applied to the coevolution of ideas on religion, science, and institutions

  2024cites this paper
• Structural topic models for the automated measurement of entrepreneurial motivation among college students of different disciplines

  2024cites this paper
• Generalized Grade-of-Membership Estimation for High-dimensional Locally Dependent Data

  2024cites this paper
• What’s in a name? The effect of named entities on topic modelling interpretability

  2024cites this paper
• Framework for understanding tourist experiences at dark heritage sites: a Cognitive-Emotional Perspective

  2024cites this paper
• To Dog-Whistle or to Bark? Elite Communication Strategies When Invoking Conspiracy Theories

  2024cites this paper
• Discovering themes and trends in electricity supply chain area research

  2023cites this paper
• Making Sense of Post-match Fan Behaviors in the Online Football Communities

  2023cites this paper
• Advancing mental health care with AI-enabled precision psychiatry tools: A patent review

  2023cites this paper
• Purpose Claims and Capacity-Based Credibility: Evidence from the Labor Market

  2023cites this paper
• Uncovering the hidden structure of dynamic T cell composition in peripheral blood during cancer immunotherapy: a topic modeling approach

  2023cites this paper
• The Materiality of Risk Factor Disclosures through a Structural Topic Model

  2023cites this paper
• Incorporating structural topic modeling into short text analysis

  2023cites this paper
• What is Cognitive Computing? An Architecture and State of The Art

  2023cites this paper
• When subordinates do not follow: A typology of subordinate resistance as perceived by leaders

  2023cites this paper
• Recent Advances in Text Analysis

  2023influential citation
• Selecting the Number and Labels of Topics in Topic Modeling: A Tutorial

  2023cites this paper
• Changing stereotypes of partisans in the Trump Era

  2023cites this paper
• CataBEEM: Integrating Latent Interaction Categories in Node-wise Community Detection Models for Network Data

  2023cites this paper
• Censor and Sensitivity: How China Handles US Embassy’s Public Diplomacy in Chinese Cyber Space

  2023cites this paper
• Incorporating an Unsupervised Text Mining Approach into Studying Logistics Risk Management: Insights from Corporate Annual Reports and Topic Modeling

  2023cites this paper
• The Adjustment of Pressure Perception in E-Government Response: A Perspective of the Political System Theory

  2023cites this paper
• A topic modeling approach reveals the dynamic T cell composition of peripheral blood during cancer immunotherapy

  2023cites this paper
• Nowcasting GDP using tone-adjusted time varying news topics: Evidence from the financial press

  2023cites this paper
• Empirical evidence of urban climate adaptation alignment with sustainable development: Application of LDA

  2023cites this paper
• A Review of Stability in Topic Modeling: Metrics for Assessing and Techniques for Improving Stability

  2023cites this paper
• Research topics and intellectual structure in Human Resource Development International 1998–2022: a bibliometric and topic modelling review

  2023cites this paper
• Selecting More Informative Training Sets with Fewer Observations

  2023cites this paper
• Attitudes toward COVID-19 Vaccination on Social Media: A Cross-Platform Analysis

  2022cites this paper
• Leaving the Institution or Leaving the Academy? Analyzing the Factors that Faculty Weigh in Actual Departure Decisions

  2022cites this paper
• Characterization of the continuous transcriptional heterogeneity in Wilms’ tumors using unsupervised machine learning

  2022cites this paper
• Transcriptional profiling of human Vδ1 T cells reveals a pathogen-driven adaptive differentiation program

  2022cites this paper
• Node-Level Community Detection within Edge Exchangeable Models for Interaction Processes

  2022cites this paper
• Tracing the evolution of service robotics: Insights from a topic modeling approach

  2022cites this paper
• A 3D transcriptomics atlas of the mouse nose sheds light on the anatomical logic of smell

  2022cites this paper
• Regional Competitiveness: A Structural-Based Topic Analysis on Recent Literature

  2022cites this paper
• What's Up With These Conversational Health Agents? From Users' Critiques to Implications for Design

  2022cites this paper
• Strategic environment effect and communication

  2022cites this paper
• Corporate Social Responsibility Activities Through Twitter: From Topic Model Analysis to Indexes Measuring Communication Characteristics

  2022cites this paper
• A one-hundred-year structural topic modeling analysis of the knowledge structure of international management research

  2022cites this paper
• Detecting Negative Campaigning on Twitter Against The Greens

  2022cites this paper
• A scoping review on the use of natural language processing in research on political polarization: trends and research prospects

  2022cites this paper
• Research Topics and Collaboration in Human Resource Development Review 2012–2021: A Bibliometrics Approach

  2022cites this paper
• Understanding information needs during COVID-19: A comparison study between an online health community and a Q&A platform

  2022cites this paper
• A new method for mining information of gut microbiome with probabilistic topic models

  2022cites this paper
• Collective response to the health crisis among German Twitter users: A structural topic modeling approach

  2022cites this paper
• A novel RORγt+ antigen presenting cell type instructs microbiota-dependent regulatory T cell differentiation and tolerance during early life

  2022cites this paper
• A novel RORgt+ antigen presenting cell type instructs microbiota-dependent regulatory T cell differentiation and tolerance during early life

  2022cites this paper
• Who Portrayed It as “The Chinese Virus”?An Analysis of the Multiplatform Partisan Framing in U.S. News Coverage About China in the COVID-19 Pandemic

  2022cites this paper
• Novel antigen-presenting cell imparts Treg-dependent tolerance to gut microbiota

  2022cites this paper
• Posterior summaries of grocery retail topic models: Evaluation, interpretability and credibility

  2022cites this paper
• Direct-to-Consumer Genetic Testing on Social Media: Topic Modeling and Sentiment Analysis of YouTube Users' Comments

  2022cites this paper
• A novel lineage of ROR g t + Aire + antigen presenting cells promotes peripheral generation of intestinal regulatory T cells and tolerance during early life

  2022cites this paper
• Three families of automated text analysis.

  2022cites this paper
• Human Factors in Phishing Attacks: A Systematic Literature Review

  2021cites this paper
• Skin-resident innate lymphoid cells converge on a pathogenic effector state

  2021cites this paper
• Automatic topic labeling using graph-based pre-trained neural embedding

  2021cites this paper
• Productivity and interdisciplinary impacts of Organized Research Units

  2021cites this paper
• Language patterns of outgroup prejudice.

  2021cites this paper
• Tracing Policy-relevant Information in Social Media: The Case of Twitter before and during the COVID-19 Crisis

  2021cites this paper
• The strategic usage of Facebook by local governments: A structural topic modelling analysis

  2021cites this paper
• Comparison of learning analytics and educational data mining: A topic modeling approach

  2021cites this paper
• 26 years left behind: a historical and predictive analysis of electronic business research

  2021cites this paper
• A Bayesian approach to modeling topic-metadata relationships

  2021cites this paper
• Non-negative matrix factorization algorithms greatly improve topic model fits

  2021cites this paper
• World Wide Web? A closer look at the transnational online public discourse on climate change

  2021cites this paper
• Stock price relevance of voluntary disclosures about blockchain technology and cryptocurrencies

  2021cites this paper
• Non-negative matrix factorization algorithms generally improve topic model fits

  2021cites this paper
• Uncovering cell identity through differential stability with Cepo

  2021cites this paper
• Fast Supervised Topic Models for Short Text Emotion Detection

  2021cites this paper
• Immigration Coverage in the Black Press and the General Audience Press: What Can Mixed Methods Reveal about Race and Immigration?

  2021cites this paper
• The Increased Effect of Elections and Changing Prime Ministers on Topics Discussed in the Australian Federal Parliament between 1901 and 2018

  2021cites this paper
• A Study on the Characteristics of Academic Topics Related to Renewable Energy Using the Structural Topic Modeling and the Weak Signal Concept

  2021cites this paper
• Detecting Topic and Sentiment Trends in Physician Rating Websites: Analysis of Online Reviews Using 3-Wave Datasets

  2021cites this paper
• MultiModalTopicExplorer: Topic modeling for exploring multi-modal data from asynchronous online conversations

  2021cites this paper
• Tracking ECB’s Communication: Perspectives and Implications for Financial Markets

  2021cites this paper
• From Christ to Compassion: The Changing Language of Pastoral Care

  2021cites this paper
• Exploring Forest Sector Research Subjects and Trends from 2000 to 2019 Using Topic Modeling

  2021cites this paper
• Cepo uncovers cell identity through differential stability

  2021cites this paper
• Automatic Topics Extraction from Crowdsourced Cyclists Near-Miss and Collision Reports Using Text Mining and Artificial Neural Networks

  2021cites this paper
• A GEOMETRY-DRIVEN LONGITUDINAL TOPIC MODEL

  2021cites this paper