# Notes on: Albanna, B. F., Hillar, C., Sohl-Dickstein, J., & DeWeese, M. R. (2012): Minimum and maximum entropy distributions for binary systems with known means and pairwise correlations

## 1 Overview

- Provide upper and lower bounds on the entropy for the
*minimum* entropy distribution over arbitrarily large collections of binary unit with any fixed set of mean values and pairwise correlations
- Minimum entropy solution has entropy scaling logarithmitcally with system size for any set of first- and second-order statistics consistent with arbitrarily large systems

## 2 Notation

- \(N\) is number of spiking neurons (binary variables)
- Each neuron \(i\) has a binary state \(s_i\)
- \(\vec{s} = \big( s_1, \dots, s_N \big) \in \left\{ 0, 1 \right\}^N\) denotes the entire ensemble
- \(\mu_i = \left\langle s_i \right\rangle\) is time-average firing rates
- \(v_{ij} = \left\langle s_i s_j \right\rangle\) is the pairwise event rates