Notes on: Neal, R. M. (1998): Annealed Importance Sampling

Table of Contents

1 Notation

  • The so-called free-energy:

    \begin{equation*} F(\mathbf{v}) = - \log \sum_{\mathbf{h}}^{} \exp \big( - E(\mathbf{v}, \mathbf{h}) \big) \end{equation*}
  • Intractable to sample from \(x \sim p(x)\) with \(p(x) \propto f(x)\)
  • Can sample from \(x \sim q(x)\) with \(q(x) \propto g(x)\)
  • \(N\) independent points \(x^{(1)}, \dots, x^{(N)} \sim q(x)\)
  • Importance weight

    \begin{equation*} w^{(i)} = \frac{f \big( x^{(i)} \big)}{g \big( x^{(i)} \big)} \end{equation*}