Awesome Topic Models

A curated list of amazing topic modelling libraries.

Contents

• Libraries & Toolkits
• Models
• Techniques
• Research Implementations
• Visualizations
• Resources
• Related awesome lists

Libraries & Toolkits

• gensim - Python library for topic modelling
• scikit-learn - Python library for machine learning
• tomotopy - Python extension for Gibbs sampling based tomoto which is written in C++
• tomoto - Ruby extension for Gibbs sampling based tomoto which is written in C++
• OCTIS - Python package to integrate, optimize and evaluate topic models
• tmtoolkit - Python topic modeling toolkit with parallel processing power
• Mallet - Java-based package for topic modeling
• TopicModel4J - Java-based package for topic modeling
• BIDMach - CPU and GPU-accelerated machine learning library
• BigARTM - Fast topic modeling platform
• TopicNet - A high-level Python interface for BigARTM library
• stm - R package for the Structural Topic Model
• RMallet - R package to interface with the Java machine learning tool MALLET
• R-lda - R package for topic modelling (LDA, sLDA, corrLDA, etc.)
• topicmodels - R package with interface to C code for LDA and CTM
• lda++ - C++ library for LDA and (fast) supervised LDA (sLDA/fsLDA) using variational inference

Models

There are huge differences in performance and scalability as well as the support of advanced features as hyperparameter tuning or evaluation capabilities.

Truncated Singular Value Decomposition (SVD) / Latent Semantic Analysis (LSA) / Latent Semantic Indexing (LSI)

• scikit-learn - Python implementation using fast randomized SVD solver or a “naive” algorithm that uses ARPACK
• gensim - Python implementation using multi-pass randomized SVD solver or a one-pass merge algorithm
• SVDlibc - C implementation of SVD by Doug Rohde
• sparsesvd - Python wrapper for SVDlibc
• BIDMach - Scala implementation of a scalable approximate SVD using subspace iteration

Non-Negative Matrix Factorization (NMF or NNMF)

• scikit-learn - Python implementation using a coordinate descent or a multiplicative update solver
• gensim - Python implementation of online NMF
• BIDMach - CPU and GPU-accelerated Scala implementation with L2 loss

Latent Dirichlet Allocation (LDA) :page_facing_up:

• scikit-learn - Python implementation using online variational Bayes inference :page_facing_up:
• lda - Python implementation using collapsed Gibbs sampling which follows scikit-learn interface :page_facing_up:
• lda-gensim - Python implementation using online variational inference :page_facing_up:
• ldamulticore-gensim - Parallelized Python implementation using online variational inference :page_facing_up:
• GibbsSamplingLDA-TopicModel4J - Java implementation using collapsed Gibbs sampling :page_facing_up:
• CVBLDA-TopicModel4J - Java implementation using collapsed variational Bayesian (CVB) inference :page_facing_up:
• Mallet - Parallelized Java implementation using Gibbs sampling :page_facing_up::page_facing_up:
• gensim-wrapper-Mallet - Python wrapper for Mallet's implementation :page_facing_up::page_facing_up:
• PartiallyCollapsedLDA - Various fast parallelized samplers for LDA, including Partially Collapsed LDA, LightLDA, Partially Collapsed Light LDA and a very efficient Polya-Urn LDA
• Vowpal Wabbit - C++ implementaion using online variational Bayes inference :page_facing_up:
• tomotopy - Python binding for C++ implementation using Gibbs sampling and different term-weighting options :page_facing_up:
• topicmodel-lib - Cython library for online/streaming LDA (Online VB, Online CVB0, Online CGS, Online OPE, Online FW, Streaming VB, Streaming OPE, Streaming FW, ML-OPE, ML-CGS, ML-FW)
• jsLDA - JavaScript implementation of LDA topic modeling in the browser
• lda-nodejs - Node.js implementation of LDA topic modeling
• lda-purescript - PureScript, browser-based implementation of LDA topic modeling
• TopicModels.jl - Julia implementation of LDA
• turicreate - C++ LDA and aliasLDA implementation with export to Apple's Core ML for use in iOS, macOS, watchOS, and tvOS apps
• MeTA - C++ implementation of (parallel) collapsed Gibbs sampling, CVB0 and SCVB
• Fugue - Java implementation of collapsed Gibbs sampling with slice sampling for hyper-parameter optimization

Hyperparameter optimization

• GA-LDA - R scripts using Genetic Algorithms (GA) for hyper-paramenter optimization, based on Panichella :page_facing_up:
• Search-Based-LDA - R scripts using Genetic Algorithms (GA) for hyper-paramenter optimization by Panichella :page_facing_up:
• Dodge - Python tuning tool that ignores redundant tunings :page_facing_up:
• LDADE - Python tuning tool using differential evolution :page_facing_up:
• ldatuning - R package to find optimal number of topics for LDA :page_facing_up:
• Scalable - Scalable Hyperparameter Selection for LDA :page_facing_up:

Evaluation

• topic_interpretability - Computation of the semantic interpretability of topics produced by topic models :page_facing_up:
• topic-coherence-sensitivity - Code to compute topic coherence for several topic cardinalities and aggregate scores across them :page_facing_up:
• topic-model-diversity - A collection of topic diversity measures for topic modeling :page_facing_up:

CPU-based high performance implementations

• LDA* - Tencent's hybrid sampler that uses different samplers for different types of documents in combination with an asymmetric parameter server :page_facing_up:
• FastLDA - C++ implementation of LDA :page_facing_up:
• dmlc - Single-and multi-threaded C++ implementations of lightLDA, F+LDA, AliasLDA, forestLDA and many more
• SparseLDA - Java algorithm and data structure for evaluating Gibbs sampling distributions used in Mallet :page_facing_up:
• warpLDA - C++ cache efficient LDA implementation which samples each token in O(1) :page_facing_up:
• lightLDA - C++ implementation using O(1) Metropolis-Hastings sampling :page_facing_up:
• F+LDA - C++ implementation of F+LDA using an appropriately modified Fenwick tree :page_facing_up:
• AliasLDA - C++ implemenation using Metropolis-Hastings and alias method:page_facing_up:
• Yahoo-LDA - Yahoo!'s topic modelling framework :page_facing_up:
• PLDA+ - Google's C++ implementation using data placement and pipeline processing :page_facing_up:
• Familia - A toolkit for industrial topic modeling (LDA, SentenceLDA and Topical Word Embedding) :warning: :page_facing_up:

GPU-based high performance implementations

• SaberLDA - GPU-based system that implements a sparsity-aware algorithm to achieve sublinear time complexity
• GS-LDA-BIDMach - CPU and GPU-accelerated Scala implementation using Gibbs sampling
• VB-LDA-BIDMach - CPU and GPU-accelerated Scala implementation using online variational Bayes inference

Hierarchical Dirichlet Process (HDP) :page_facing_up:

• gensim - Python implementation using online variational inference :page_facing_up:
• tomotopy - Python extension for C++ implementation using Gibbs sampling :page_facing_up:
• Mallet - Java-based package for topic modeling using Gibbs sampling
• TopicModel4J - Java implementation using Gibbs sampling based on Chinese restaurant franchise metaphor
• hca - C implementation using Gibbs sampling with/without burstiness modelling
• bnp - Cython reimplementation based on online-hdp following scikit-learn's API.
• Scalable HDP - interesting paper

Hierarchical LDA (hLDA) :page_facing_up:

• tomotopy - Python extension for C++ implementation using Gibbs sampling
• Mallet - Java implementation using Gibbs sampling
• hlda - Python package based on Mallet's Gibbs sampler having a fixed depth on the nCRP tree
• hLDA - C implementation of hierarchical LDA by David Blei

Dynamic Topic Model (DTM) :page_facing_up:

• tomotopy - Python extension for C++ implementation using Gibbs sampling based on FastDTM
• FastDTM - Scalable C++ implementation using Gibbs sampling with Stochastic Gradient Langevin Dynamics (MCMC-based) :page_facing_up:
• ldaseqmodel-gensim - Python implementation using online variational inference :page_facing_up:
• dtm-BigTopicModel - C++ engine for running large-scale topic models
• tca - C implementation using Gibbs sampling with/without burstiness modelling :page_facing_up:
• DETM - Python implementation of the Dynamic Embedded Topic Model :page_facing_up:

Author-topic Model (ATM) :page_facing_up:

• gensim - Python implementation with online training (constant in memory w.r.t. the number of documents)
• TopicModel4J - Java implementation
• Matlab Topic Modeling Toolbox - Matlab and C++ implementation using Gibbs sampling
• Topic-Model - Simple Python implementation using Gibbs sampling

Labeled Latent Dirichlet Allocation (LLDA, Labeled-LDA, L-LDA) :page_facing_up:

• tomotopy - Python extension for C++ implementation using Gibbs sampling
• TopicModel4J - Java implementation
• Mallet - Java implementation using Gibbs sampling :page_facing_up:
• gensims_mallet_wrapper - Python wrapper for Mallet using gensim interface
• STMT - Scala implementation by Daniel Ramage
• topbox - Python wrapper for labeled LDA implementation of Stanford TMT
• Labeled-LDA-Python - Python implementation (easy to use, does not scale)
• JGibbLabeledLDA - Java implementation based on the popular JGibbLDA package

Partially Labeled Dirichlet Allocation (PLDA) / Dirichlet Process (PLDP) :page_facing_up:

• tomotopy - Python extension for C++ implementation using Gibbs sampling
• TopicModel4J - Java implementation using collapsed Gibbs sampling
• STMT - Scala implementation of PLDA & PLDP by Daniel Ramage

Dirichlet Multinomial Regression (DMR) topic model :page_facing_up:

• tomotopy - Python extension for C++ implementation using Gibbs sampling
• Mallet - Java-based package for topic modeling

Generalized Dirichlet Multinomial Regression (g-DMR) topic model :page_facing_up:

• tomotopy - Python extension for C++ implementation using Gibbs sampling

Link LDA

• PTM - implemented as benchmark :page_facing_up:
• TopicModel4J - Java implementation using collapsed Gibbs sampling

Correlated Topic Model (CTM) a.k.a. logistic-normal topic models

• tomotopy - Python extension for C++ implementation using Gibbs sampling :page_facing_up:
• ctm-c - Original C implementation of the correlated topic model by David Blei :page_facing_up:
• BigTopicModel - C++ engine for running large-scale DTM :page_facing_up:
• stm - R package for the Structural Topic Model (CTM in case of no covariates) :page_facing_up:

Relational Topic Model (RTM)

• BigTopicModel - C++ engine for running large-scale topic models
• Constrained-RTM - Java implementation of Contrained RTM :page_facing_up:
• R-lda - R implementation using collapsed Gibbs sampling

Supervised LDA (sLDA) :page_facing_up:

• tomotopy - Python extension for C++ implementation using Gibbs sampling
• R-lda - R implementation using collapsed Gibbs sampling
• slda - Cython implementation of Gibbs sampling for LDA and various sLDA variants
  • supervised LDA (linear regression)
  • binary logistic supervised LDA (logistic regression)
  • binary logistic hierarchical supervised LDA (trees)
  • generalized relational topic models (graphs)
• YWWTools - Java implementation using Gibbs sampling for LDA and various sLDA variants:
  • BS-LDA: Binary SLDA
  • Lex-WSB-BS-LDA: BS-LDA with Lexcial Weights and Weighted Stochastic Block Priors
  • Lex-WSB-Med-LDA: Lex-WSB-BS-LDA with Hinge Loss
• sLDA - C++ implementation of supervised topic models with a categorical response

Topic Models for short documents

Sentence-LDA / SentenceLDA / Sentence LDA :page_facing_up:

• TopicModel4J - Java implementation of Sentence-LDA using collapsed Gibbs sampling
• Familia - Apply inference on pre-trained SentenceLDA models :warning: :page_facing_up:

Dirichlet Multinomial Mixture Model (DMM) :page_facing_up:

• GPyM_TM - Python implementation of DMM and Poisson model
• TopicModel4J - Java implementation using collapsed Gibbs sampling :page_facing_up:
• jLDADMM - Java implementation using collapsed Gibbs sampling :page_facing_up:

Dirichlet Process Multinomial Mixture Model (DPMM)

• TopicModel4J - Java implementation using collapsed Gibbs sampling :page_facing_up:

Pseudo-document-based Topic Model (PTM) :page_facing_up:

• tomotopy - Python extension for C++ implementation using Gibbs sampling
• TopicModel4J - Java implementation using collapsed Gibbs sampling

Biterm topic model (BTM)

• TopicModel4J - Java implementation using collapsed Gibbs sampling
• BTM - Original C++ implementation using collapsed Gibbs sampling :page_facing_up:
• BurstyBTM - Original C++ implementation of the Bursty BTM (BBTM) :page_facing_up:
• OnlineBTM - Original C++ implementation of online BTM (oBTM) and incremental BTM (iBTM) :page_facing_up
• R-BTM - R package wrapping the C++ code from BTM

Others

• STTM - Java implementation and evaluation of DMM, WNTM, PTM, ETM, GPU-DMM, GPU-DPMM, LF-DMM :page_facing_up:
• SATM - Java implementation of Self-Aggregation Topic Model :page_facing_up:
• shorttext - Python implementation of various algorithms for Short Text Mining

Miscellaneous topic models

• trLDA - Python implementation of streaming LDA based on trust-regions :page_facing_up:
• Logistic LDA - Tensorflow implementation of Discriminative Topic Modeling with Logistic LDA :page_facing_up:
• EnsTop - Python implementation of ENSemble TOPic modelling with pLSA
• Dual-Sparse Topic Model - implemented in TopicModel4J using collapsed variational Bayes inference :page_facing_up:
• Multi-Grain-LDA - MG-LDA implemented in tomotopy using collapsed Gibbs sampling :page_facing_up:
• lda++ - C++ library for LDA and (fast) supervised LDA (sLDA/fsLDA) using variational inference :page_facing_up: :page_facing_up:
• discLDA - C++ implementation of discLDA based on GibbsLDA++ :page_facing_up:
• GuidedLDA - Python implementation that can be guided by setting some seed words per topic (using Gibbs sampling) :page_facing_up:
• seededLDA - R package that implements seeded-LDA for semi-supervised topic modeling
• keyATM - R package for Keyword Assisted Topic Models.
• hca - C implementation of non-parametric topic models (HDP, HPYP-LDA, etc.) with focus on hyperparameter tuning
• BayesPA - Python interface for streaming implementation of MedLDA, maximum entropy discrimination LDA (max-margin supervised topic model) :page_facing_up:
• sailing-pmls - Parallel LDA and medLDA implementation
• BigTopicModel - C++ engine for running large-scale MedLDA models :page_facing_up:
• DAPPER - Python implementation of Dynamic Author Persona (DAP) topic model :page_facing_up:
• ToT - Python implementation of Topics Over Time (A Non-Markov Continuous-Time Model of Topical Trends) :page_facing_up:
• MLTM - C implementation of multilabel topic model (MLTM) :page_facing_up:
• sequence-models - Java implementation of block HMM and the mixed membership Markov model (M4)
• Entropy-Based Topic Modeling - Java implementation of Entropy-Based Topic Modeling on Multiple Domain-Specific Text Collections
• ST-LDA - ST-LDA: Single Topic LDA :page_facing_up:
• MTM - Java implementation of Multilingual Topic Model :page_facing_up:
• YWWTools - Java-based package for various topic models by Weiwei Yang

Exotic models

• TEM - Topic Expertise Model :page_facing_up:
• PTM - Prescription Topic Model for Traditional Chinese Medicine Prescriptions :page_facing_up: (interesting benchmark models)
• KGE-LDA - Knowledge Graph Embedding LDA :page_facing_up:
• LDA-SP - A Latent Dirichlet Allocation Method for Selectional Preferences :page_facing_up:
• LDA+FFT - LDA and FFTs (Fast and Frugal Trees) for better comprehensibility :page_facing_up:

Embedding based Topic Models

• BERTopic - BERTopic supports guided, (semi-) supervised, and dynamic topic modeling and visualization :page_facing_up:
• CTM - CTMs combine contextualized embeddings (e.g., BERT) with topic models
• ETM - Embedded Topic Model :page_facing_up:
• D-ETM - Dynamic Embedded Topic Model :page_facing_up:
• ProdLDA - Original TensorFlow implementation of Autoencoding Variational Inference (AEVI) for Topic Models :page_facing_up:
• pytorch-ProdLDA - PyTorch implementation of ProdLDA :page_facing_up:
• CatE - Discriminative Topic Mining via Category-Name Guided Text Embedding :page_facing_up:
• Top2Vec - Python implementation that learns jointly embedded topic, document and word vectors :page_facing_up:
• lda2vec - Mixing dirichlet topic models and word embeddings to make lda2vec :page_facing_up:
• lda2vec-pytorch - PyTorch implementation of lda2vec
• G-LDA - Java implementation of Gaussian LDA using word embeddings :page_facing_up:
• MG-LDA - Python implementation of (Multi-lingual) Gaussian LDA :page_facing_up:
• MetaLDA - Java implementation using Gibbs sampling that leverages document metadata and word embeddings :page_facing_up:
• LFTM - Java implementation of latent feature topic models (improving LDA and DMM with word embeddings) :page_facing_up:
• CorEx - Recover latent factors with Correlation Explanation (CorEx) :page_facing_up:
• Anchored CorEx - Hierarchical Topic Modeling with Minimal Domain Knowledge :page_facing_up:
• Linear CorEx - Latent Factor Models Based on Linear Total CorEx :page_facing_up:

Probabilistic Programming Languages (PPL) (a.k.a. Build your own Topic Model)

• Stan - Platform for statistical modeling and high-performance statistical computation, e.g., LDA :page_facing_up:
• PyMC3 - Python package for Bayesian statistical modeling and probabilistic machine learning, e.g., LDA :page_facing_up:
• Turing.jl - Julia library for general-purpose probabilistic programming :page_facing_up:
• TFP - Probabilistic reasoning and statistical analysis in TensorFlow, e.g., LDA :page_facing_up:
• edward2 - Simple PPL with core utilities in the NumPy and TensorFlow ecosystem :page_facing_up:
• pyro - PPL built on PyTorch, e.g., prodLDA :page_facing_up:
• edward - A PPL built on TensorFlow, e.g., LDA :page_facing_up:
• ZhuSuan - A PPL for Bayesian deep learning, generative models, built on Tensorflow, e.g., LDA :page_facing_up:

Research Implementations

• lda-c - C implementation using variational EM by David Blei
• sLDA - C++ implementation of supervised topic models with a categorical response.
• onlineldavb - Python online variational Bayes implementation by Matthew Hoffman :page_facing_up:
• HDP - C++ implementation of hierarchical Dirichlet processes by Chong Wang
• online-hdp - Python implementation of online hierarchical Dirichlet processes by Chong Wang
• ctr - C++ implementation of collaborative topic models by Chong Wang
• dtm - C implementation of dynamic topic models by David Blei & Sean Gerrish
• ctm-c - C implementation of the correlated topic model by David Blei
• diln - C implementation of Discrete Infinite Logistic Normal (with HDP option) by John Paisley
• hLDA - C implementation of hierarchical LDA by David Blei
• turbotopics - Python implementation that finds significant multiword phrases in topics by David Blei
• Stanford Topic Modeling Toolbox - Scala implementation of LDA, labeledLDA, PLDA, PLDP by Daniel Ramage and Evan Rosen
• LDAGibbs - Java implementation of LDA using Gibbs sampling by Liu Yang
• Matlab Topic Modeling Toolbox - Matlab implementations of LDA, ATM, HMM-LDA, LDA-COL (Collocation) models by Mark Steyvers and Tom Griffiths
• cvbLDA - Python C extension implementation of collapsed variational Bayesian inference for LDA
• fast - A Fast And Scalable Topic-Modeling Toolbox (Fast-LDA, CVB0) by Arthur Asuncion and colleagues :page_facing_up:

Popular Implementations (but not maintained anymore)

• Stanford Topic Modeling Toolbox - Scala implementation of LDA, labeledLDA, PLDA, PLDP by Daniel Ramage and Evan Rosen
• Matlab Topic Modeling Toolbox - Matlab implementations of LDA, ATM, HMM-LDA, LDA-COL (Collocation) models by Mark Steyvers and Tom Griffiths
• GibbsLDA++ - C++ implementation using Gibbs sampling :page_facing_up: :fork_and_knife:
• JGibbLDA - Java implementation using Gibbs sampling
• Mr.LDA - Scalable Topic Modeling using Variational Inference in MapReduce :page_facing_up:

Learning Implementations (hopefully easy to understand)

• topic_models - Python implementation of LSA, PLSA and LDA
• Topic-Model - Python implementation of LDA, Labeled LDA, ATM, Temporal Author-Topic Model using Gibbs sampling

Visualizations

• LDAvis - R package for interactive topic model visualization
• pyLDAvis - Python library for interactive topic model visualization
• scalaLDAvis - Scala port of pyLDAvis
• dtmvisual - Python package for visualizing DTM (trained with gensim)
• TMVE online - Online Django variant of topic model visualization engine (TMVE)
• TMVE - Original topic model visualization engine (LDA trained with lda-c) :page_facing_up:
• topicmodel-lib - Python wrapper for TMVE for visualizing LDA (trained with topicmodel-lib)
• wordcloud - Python package for visualizing topics via word_cloud
• Mallet-GUI - GUI for creating and analyzing topic models produced by MALLET
• TWiC - Topic Words in Context is a highly-interactive, browser-based visualization for MALLET topic models
• dfr-browser - Explore Mallet's topic models of texts in a web browser
• Termite - Explore topic models using term-topic matrix, group-in-a-box visualization or scatter plot.
• Topics - Python library for topic modeling and visualization
• TopicsExplorer - Explore your own text collection with a topic model – without prior knowledge :page_facing_up:
• topicApp - A Simple Shiny App for Topic Modeling
• stminsights - A Shiny Application for Inspecting Structural Topic Models

Dirichlet hyperparameter optimization techniques

• Slice sampling
• Minka
  • fastfit
  • dirichlet Python port of fastfit
  • lightspeed
  • lecture-notes
• Newton-Raphson Method
• fixed-point iteration - Wallach's PhD thesis, chapter 2.3

Resources

• David Blei - David Blei's Homepage with introductory materials

Related awesome lists

• awesome-machine-learning
• awesome-datascience
• awesome-python-data-science

Contribute

Contributions welcome! Read the contribution guidelines first.

License

To the extent possible under law, Jonathan Schneider has waived all copyright and related or neighboring rights to this work.