Conditional Densities: A Bayesian Framework

In Bayesian inference, the probability density function of a vector ${\bf z}$ is typically estimated from a training set of such vectors ${\cal Z}$ as shown in Equation 5.1 [5].

$$p({\bf z} \vert {\cal Z} ) = \int p( {\bf z}, \Theta \vert {\... ...bf z} \vert \Theta ,{\cal Z}) p(\Theta \vert {\cal Z}) d\Theta$$ (5.1)

By integrating over $\Theta$, we are essentially integrating over all the pdf models possible. This involves varying the families of pdfs and all their parameters. However, often, this is impossible and instead a family is selected and only its parametrization $\Theta$ is varied. Each $\Theta$ is a parametrization of the pdf of ${\bf z}$ and is weighted by its likelihood given the training set. However, computing the integral ^5.1 is not always straightforward and Bayesian inference is approximated via maximum a posteriori (MAP) or maximum likelihood (ML) estimation as in Equation 5.2. The EM algorithm is frequently utilized to perform these maximizations.

 

$$p({\bf z} \vert {\cal Z} ) \approx p({\bf z} \vert \Theta^\ast , ... ...mbox{MAP} \\ \arg\max p( {\cal Z} \vert \Theta) & \mbox{ML} \end{array}\right.$$ (5.2)