Dirichlet_processes
Dirichlet_processes copied to clipboard
Brief introduction and implementations of related concepts to Dirichlet Processes: GEM distribution, Polya Urn, Chinese restaurant process, Stick-Breaking construction, and Posterior of a DP.
Dirichlet Process
Summary
Brief introduction and implementations of related concepts to Dirichlet Processes: GEM distribution, Polya Urn, Chinese restaurant process, Stick-Breaking construction, and Posterior of a DP.- Griffiths-Engen-McCloskey (GEM) Distribution:
- Definition.
- Function to construct samples
- Function to construct sample distribution
- GEM Figures for different values:
- Figure 1: Distribution of weight values for each dimension of , shows the behavior of for different values
- Figure 2: Sample vectors and the decreasing trend in average of the weights as we increase dimension
- Figure 3: Stick representation for 15 samples, from the whole probability vector show each dimension weight with a different color. Another way of representing Figure 1.
- Polya urn.
- Definition.
- Function to model Polya urn.
- Figure 1: Distribution of independent samples of Polya urns, only high number of draws show to be distributed as a Beta distribution.
- Chinese Restaurant process.
- Definition.
- Function to model table assignations.
- Function to construct the Chinese restaurant process
- Figure 1: CRP mean and variance on number of tables for different values.
- Dirichlet Process:.
- Definitions:
- Stick-breaking representation.
- Ferguson's definition.
- Function to construct samples using the stick-breaking representation:
- Function to construct sample distribution
- DP Figures for different values:
- Figure 1: Draws from a DP using the stick-breaking representation.
- Figure 2: Visualization of a DP through Ferguson's definition: , , and the impact of the concentration parameter .
- Figure 3: Visualization of a DP through Ferguson's definition: Cumulative distributions of the random probability measure and the base measure .
- Posterior DP:
- Posterior construction through stick-breaking.
- Function to construct the DP posterior from obseravations.
- Figure 1: DP posterior visualizations for different number of observations of an 'assumed true' DP posterior and values in the prior.
Notebooks:
- DP Visualizations
- Variational Inference GMM and DPMM