Approximate Bayesian Computation (ABC)

Table of Contents

Notation

  • approximate_bayesian_computation_78504c98de3602d3ca0f9c9e1bc9f50301460233.png parameters
  • approximate_bayesian_computation_c3f64048c5805d9a401423a533acbb0372039c00.png generated samples from model with parameters approximate_bayesian_computation_e7a86163f39fa21d4a2ed66946369cdeb900ef42.png
  • approximate_bayesian_computation_688e6e5747f484cd8a4e57dfea18e23bd317e255.png denotes observed data
  • approximate_bayesian_computation_78fe10aaa74d6bed9ab2238be53774fe93bfbf0e.png is the domain of the observations
  • approximate_bayesian_computation_9ab766fe2c081b82a304866d50f951fadbdb5704.png is a metric on approximate_bayesian_computation_78fe10aaa74d6bed9ab2238be53774fe93bfbf0e.png
  • approximate_bayesian_computation_78bb582c5738c8937eda79522ab2d8814eba0be7.png

Overview

  • Cases where computing the likelihood of the observed data approximate_bayesian_computation_688e6e5747f484cd8a4e57dfea18e23bd317e255.png is intractable
  • ABC uses approximation of the likelihood obtained from simulation

Rejection ABC

Let approximate_bayesian_computation_95253ea0c2082f08ac36eed736be50b9577033e8.png be a similarity threshold, and approximate_bayesian_computation_9ab766fe2c081b82a304866d50f951fadbdb5704.png be the notion of distance, e.g. premetric on domain approximate_bayesian_computation_78fe10aaa74d6bed9ab2238be53774fe93bfbf0e.png of observations.

The rejection ABC proceeds as follows:

  1. Sample multiple model parameters approximate_bayesian_computation_b295fcfa420094d7f3941ef9e6865b119bf379fe.png.
  2. For each approximate_bayesian_computation_e7a86163f39fa21d4a2ed66946369cdeb900ef42.png, generate psuedo-dataset approximate_bayesian_computation_a3a7f43f807b9e381fc50e0fab140c0df0a03e17.png from approximate_bayesian_computation_c3f64048c5805d9a401423a533acbb0372039c00.png
  3. For each psuedo-datset approximate_bayesian_computation_a3a7f43f807b9e381fc50e0fab140c0df0a03e17.png, if approximate_bayesian_computation_0e21838877590cac337d8fb0805ea5866b451b94.png, accept the generated approximate_bayesian_computation_a3a7f43f807b9e381fc50e0fab140c0df0a03e17.png, otherwise reject approximate_bayesian_computation_a3a7f43f807b9e381fc50e0fab140c0df0a03e17.png.

Result: Exact sample approximate_bayesian_computation_5b0379fae7a71b20150b08c5490e9ac52831b544.png from approximated posterior approximate_bayesian_computation_a8c37f3fc0ece3b666eda96d40f7723572783882.png, where

approximate_bayesian_computation_e1fd11b6dfbda81b28d23869bcaec9f81f2a725d.png

Choice of approximate_bayesian_computation_9ab766fe2c081b82a304866d50f951fadbdb5704.png is crucial in the design of a n accurate ABC algorithm.

Soft ABC

One can interpret the approximate likelihood approximate_bayesian_computation_d706c2f2169fb7647e04dd6d70e039b060a5a2a6.png in rejection ABC as the convolution of the true likelihood approximate_bayesian_computation_c3f64048c5805d9a401423a533acbb0372039c00.png and the "similarity" kernel approximate_bayesian_computation_ac2acc6c169b2ea7020ce4a767228a515ae8ed2f.png

approximate_bayesian_computation_76c4f2c60faa670b65c6b92eaa410079b7aa6b52.png

In fact, one can use any similarity kernel parametrised by approximate_bayesian_computation_03d96c52df5ea6a510607e2260e0745d11e4bd85.png satisfying

approximate_bayesian_computation_0d44f6ce46622eddd6a07e77fbfdc1101066bf49.png

which gives rise to the Soft ABC methods:

Soft ABC is an extension of rejection ABC which instead weights the parameter samples from the model instead of rejecting or accepting.

An example is using the Gaussian kernel:

approximate_bayesian_computation_fcebe90b97b804acfb37b469359be596f355739b.png

Which results in the weighted sample

approximate_bayesian_computation_b851896963f5411c576d2ca4b22488409df85e9c.png

which can be directly utilized in estimating posterior expectations, i.e. for a test function approximate_bayesian_computation_cdd1cc131da6040eca078917132a377727053c44.png

approximate_bayesian_computation_6c1642a341feaf9a6b06c27c4de48c393acb19d5.png