Beta and Dirichlet Distributions

Beta and Dirichlet Distributions

Beta Distribution

Beta distribution is frequently used as a prior.

Dirichlet Distribution and Dirichlet Process

Dirichlet distribution is a multinomial generalization of beta distribution.. A perfect example of this phenomenon comes from stackexchange:  A biased (weighted) die can be described by a multinomial distribution. A machine that makes biased die with some random error can be described by a Dirichlet distribution–a distribution over distributions.
\begin{eqnarray}
f(x_1,\ldots,x_n;\alpha_1,\ldots,\alpha_n)=\frac{1}{B(\alpha_1,\ldots,\alpha_n)} \prod_{i=1}^n x_i ^{\alpha_i-1}
\end{eqnarray}
where
\begin{eqnarray}
x_1+\ldots+x_n=1.
\end{eqnarray}
and $\alpha_k>0$

Dirichlet distribution is a probability distribution over all possible multinomial distributions. It represents the uncertainty over the actual distribution that created the row.

Dirichlet Process

The Dirichlet process can be seen as the infinite-dimensional generalization of the Dirichlet distribution. It has two parameters: $\alpha$ and $G$. $\alpha$ is called the concentration parameter. $G$ is the base distribution.

\begin{eqnarray}
x \sim Multinomial(DP(\alpha, G))
\end{eqnarray}
Note that $\Sigma x =1$, ie, the dirichlet distribution is defined over simplexes.

Dirichlet process and dirichlet distribution are different.

Dirichlet as stick breaking problem