Notes on: Albanna, B. F., Hillar, C., Sohl-Dickstein, J., & DeWeese, M. R. (2012): Minimum and maximum entropy distributions for binary systems with known means and pairwise correlations

Table of Contents

1 Overview

  • Provide upper and lower bounds on the entropy for the minimum entropy distribution over arbitrarily large collections of binary unit with any fixed set of mean values and pairwise correlations
  • Minimum entropy solution has entropy scaling logarithmitcally with system size for any set of first- and second-order statistics consistent with arbitrarily large systems

2 Notation

  • \(N\) is number of spiking neurons (binary variables)
  • Each neuron \(i\) has a binary state \(s_i\)
  • \(\vec{s} = \big( s_1, \dots, s_N \big) \in \left\{ 0, 1 \right\}^N\) denotes the entire ensemble
  • \(\mu_i = \left\langle s_i \right\rangle\) is time-average firing rates
  • \(v_{ij} = \left\langle s_i s_j \right\rangle\) is the pairwise event rates