Notes on: Arjovsky, M., Chintala, S., & Bottou, L\'eon (2017): Wasserstein Gan

Table of Contents

Notation

  • parametric family of densities arjovsky17_wasser_gan_8258681db09612bd989f5631e1a9a5437493bfc2.png
  • arjovsky17_wasser_gan_d1421a1a9ba3882dd190d4dd6970007d8281e388.png is real data examples
  • arjovsky17_wasser_gan_f6a60ab02c0fba1d6edd7a0d22c7ca90dbdd55a0.png denotes real data distribution
  • arjovsky17_wasser_gan_d01455395d76d0cfa9daf8e36ffd8b30a3808917.png denotes distribution of the parametrized density arjovsky17_wasser_gan_e6c03b4eb92a2e614a0c3b8448a35245b0e5c5f9.png
  • arjovsky17_wasser_gan_76879b948635123ecd29d7cd65a1060145ba040e.png a compact metric space (e.g. space of images arjovsky17_wasser_gan_8992410cead5e1a6c48da23f1603dc344ea91f99.png)
  • arjovsky17_wasser_gan_bc4d8c2b4155ba3e3f727e5dc3041a210b747a1a.png denotes the set of all Borel sets of arjovsky17_wasser_gan_76879b948635123ecd29d7cd65a1060145ba040e.png, i.e. arjovsky17_wasser_gan_bc4d8c2b4155ba3e3f727e5dc3041a210b747a1a.png forms a Borel sigma-algebra
  • arjovsky17_wasser_gan_5f6db8a832e22d1ac2af4607538f9bfb1de306d5.png denotes the space of probability measures defined on arjovsky17_wasser_gan_76879b948635123ecd29d7cd65a1060145ba040e.png
  • arjovsky17_wasser_gan_a5328852735b02ca3cb57e7ad57ab10b50d59567.png
  • arjovsky17_wasser_gan_3d69dc96bbaa2f683e1983ae8bd40d27ed7ff258.png is the total variation (TV) distance
  • arjovsky17_wasser_gan_0aa0470120f07b61ef9e985225cf9bbd6b79177c.png is a random variable with a fixed distribution arjovsky17_wasser_gan_1a0992ed24a704eed765cdac778e5c7db57bbe13.png
  • arjovsky17_wasser_gan_7d999f60cefe04a92fa51238b66949450259f548.png is the parametric function (e.g. a neural network) which generates samples following a certain distribution arjovsky17_wasser_gan_d01455395d76d0cfa9daf8e36ffd8b30a3808917.png
  • arjovsky17_wasser_gan_661c791039d54369a2013d9aecb898e6bdf88a3c.png is the weight space, i.e. arjovsky17_wasser_gan_692c4fdbe2aa0562f4b55cc4ed28503e48ba8921.png, and is a compact set
  • critic refers to the arjovsky17_wasser_gan_b951bdb2b9d81cb608d83545a90db2d1a49469b6.png, a Lipschitz-continuous function used in the Kantorovhich-Rubinstein duality to make the problem of minimzing Earth-Mover distance between arjovsky17_wasser_gan_f6a60ab02c0fba1d6edd7a0d22c7ca90dbdd55a0.png and arjovsky17_wasser_gan_9995e32970c1b648fc9ca38ca654b76a9fe762b7.png tractable
  • discriminator refers to the arjovsky17_wasser_gan_b689cba8d7566f6adaf605a844e193a27e155078.png in a "standard" GAN
  • arjovsky17_wasser_gan_9f7dd90c40d29155f306eaf5350f8cb2493b3ecc.png with arjovsky17_wasser_gan_a1928854a3ed5ca1bc1e8fb77465f490a76282f2.png since arjovsky17_wasser_gan_cdd1cc131da6040eca078917132a377727053c44.png is bounded
  • arjovsky17_wasser_gan_713439ec4189ebc79a1bfab127d8c10961099122.png is a normed vector space
  • arjovsky17_wasser_gan_6aabe50a1d23b6d130ce2492104c468d017b7ab0.png with dual norm

    arjovsky17_wasser_gan_7e9506646420276b908d776844b226ee6c4de174.png

  • arjovsky17_wasser_gan_b261bda07708be4b381f6b4bfaf20806be6ec81b.png is the normed space of the dual
  • arjovsky17_wasser_gan_8700f80c4895c66c1fb785c5b0cf5c4b6f532c75.png is a signed measure over arjovsky17_wasser_gan_76879b948635123ecd29d7cd65a1060145ba040e.png

Motivation

Why Wasserstein is indeed weak