Approximate Bayesian Computation (ABC)
Table of Contents
Notation
parameters
generated samples from model with parameters
denotes observed data
is the domain of the observations
is a metric on
Overview
- Cases where computing the likelihood of the observed data
is intractable
- ABC uses approximation of the likelihood obtained from simulation
Rejection ABC
Let be a similarity threshold, and
be the notion of distance, e.g. premetric on domain
of observations.
The rejection ABC proceeds as follows:
- Sample multiple model parameters
.
- For each
, generate psuedo-dataset
from
- For each psuedo-datset
, if
, accept the generated
, otherwise reject
.
Result: Exact sample from approximated posterior
, where
![\begin{equation*}
\tilde{p}_{\varepsilon} \big( y^* \mid \theta \big) = \int_{B_{\varepsilon}(y^*)} p \big( y \mid \theta \big) \ dy, \qquad B_{\varepsilon}(y^*) = \{ y : \rho(y, y^*) < \varepsilon \}
\end{equation*}](../../assets/latex/approximate_bayesian_computation_225005b7ffcbeae85fa223be2f5779a0ee032e26.png)
Choice of is crucial in the design of a n accurate ABC algorithm.
Soft ABC
One can interpret the approximate likelihood in rejection ABC as the convolution of the true likelihood
and the "similarity" kernel
![\begin{equation*}
k_{\varepsilon}(y, y^*) = \mathbb{1} \big( y \in B_{\varepsilon}(y^*) \big)
\end{equation*}](../../assets/latex/approximate_bayesian_computation_da3f02dad7de4f9fcbd097a64da3c2718769ab71.png)
In fact, one can use any similarity kernel parametrised by satisfying
![\begin{equation*}
k_{\varepsilon}(y, y^*) \to \delta_{y^*}(y) \quad \text{as} \quad \varepsilon \to 0
\end{equation*}](../../assets/latex/approximate_bayesian_computation_08b635769af06ec17a1fbe620b6beea18c3e8eb1.png)
which gives rise to the Soft ABC methods:
Soft ABC is an extension of rejection ABC which instead weights the parameter samples from the model instead of rejecting or accepting.
An example is using the Gaussian kernel:
![\begin{equation*}
k_{\varepsilon}(y, y') := \exp \Bigg( - \frac{\rho^q(y, y')}{\varepsilon} \Bigg), \quad q > 0
\end{equation*}](../../assets/latex/approximate_bayesian_computation_d74992cf8fda0196426d554735e2a0772824b783.png)
Which results in the weighted sample
![\begin{equation*}
\Big\{ (\theta_j ,w_j) \Big\}_{j = 1}^M \quad \text{with} \quad w_j = \frac{k_{\varepsilon}(y_j, y^*)}{\sum_{i=1}^{M} k_{\varepsilon}(y_i, y^*)}
\end{equation*}](../../assets/latex/approximate_bayesian_computation_313d833c70942472424837a54db3edb40524ab91.png)
which can be directly utilized in estimating posterior expectations, i.e. for a test function
![\begin{equation*}
\mathbb{\hat{E}} \big[ f(\theta) \big] = \sum_{i=1}^{M} w_j f(\theta_j)
\end{equation*}](../../assets/latex/approximate_bayesian_computation_fb1a65361822bd3a8a9eb9d66fb374dcf23af647.png)