41
3.2 Background
The probabilistic model in Section 3.4 is a Dirichlet Process Mixture (DPM) model.
The objectives of this section is, first, to review the DPM and, second, to show poste-
rior simulation and posterior predictive simulation for a future observation. With the
first objective in mind, we review the Dirichlet distribution, the Dirichlet Process and
its basic properties. To explain the main ideas of posterior and posterior predictive
simulation we review the Gibbs sampling scheme proposed by MacEachern (1994).
Finally, we give an example of an application in density estimation of the DPM.
3.2.1 Dirichlet Distribution
The Dirichlet distribution is the multivariate extension of the beta distribution and
is the conjugate prior model for the parameters of a multinomial model.
Let Ga(a, β) denote the gamma distribution with shape parameter a ≥ 0 and
scale β > 0. Define Ga(a = 0,/3) as a point mass at zero. For a > 0 the gamma pdf
is
Ga(z I a, β) = ɪ exp(-z∕β)za~‰rx^z), (3.1)
i yczjp
where Is (г) denotes the indicator function of the set S.
The Dirichlet distribution is defined with all its parameters positive (see Wilks,
1962). Ferguson (1973) extends it to allow some, but not all, parameters to be equal
to zero. Let Zi,..., Zk be independent random Ga(aj, 1) variables respectively, with
a} > 0 for every j and <¾ > 0 for some j, j = 1,..., к. The Dirichlet distribution
with parameters ɑɪ,..., o⅛, here denoted by Dirichlet(aɪ,..., afc), is defined as the
distribution of the vector (Y1,..., Yk), where
k
Yj = Zj∕^Zi for J = 1,..., fc. (3.2)
г=1
When using the notation Dirichlet(aι,..., α⅛) it is assumed that α7∙ ≥ 0 for all j, and
aj > 0 for some j.