Maximally informative dimensions is a dimensionality reduction technique used in the statistical analyses of neural responses. Specifically, it is a way of projecting a stimulus onto a low-dimensional subspace so that as much information as possible about the stimulus is preserved in the neural response. It is motivated by the fact that natural stimuli are typically confined by their statistics to a lower-dimensional space than that spanned by white noise[1] but correctly identifying this subspace using traditional techniques is complicated by the correlations that exist within natural images. Within this subspace, stimulus-response functions may be either linear or nonlinear. The idea was originally developed by Tatyana Sharpee, Nicole C. Rust, and William Bialek in 2003.[2]
Mathematical formulation
Neural stimulus-response functions are typically given as the probability of a neuron generating an action potential, or spike, in response to a stimulus . The goal of maximally informative dimensions is to find a small relevant subspace of the much larger stimulus space that accurately captures the salient features of . Let denote the dimensionality of the entire stimulus space and denote the dimensionality of the relevant subspace, such that . We let denote the basis of the relevant subspace, and the projection of onto . Using Bayes' theorem we can write out the probability of a spike given a stimulus:
where
is some nonlinear function of the projected stimulus.
In order to choose the optimal , we compare the prior stimulus distribution with the spike-triggered stimulus distribution using the Shannon information. The average information (averaged across all presented stimuli) per spike is given by
- .[3]
Now consider a dimensional subspace defined by a single direction . The average information conveyed by a single spike about the projection is
- ,
where the probability distributions are approximated by a measured data set via and , i.e., each presented stimulus is represented by a scaled Dirac delta function and the probability distributions are created by averaging over all spike-eliciting stimuli, in the former case, or the entire presented stimulus set, in the latter case. For a given dataset, the average information is a function only of the direction . Under this formulation, the relevant subspace of dimension would be defined by the direction that maximizes the average information .
This procedure can readily be extended to a relevant subspace of dimension by defining
and
and maximizing .
Importance
Maximally informative dimensions does not make any assumptions about the Gaussianity of the stimulus set, which is important, because naturalistic stimuli tend to have non-Gaussian statistics. In this way the technique is more robust than other dimensionality reduction techniques such as spike-triggered covariance analyses.
References
- ^ D.J. Field. "Relations between the statistics of natural images and the response properties of cortical cells." J. Opt. Soc. am. A 4:2479-2394, 1987.
- ^ Sharpee, Tatyana, Nicole C. Rust, and William Bialek. Maximally informative dimensions: analyzing neural responses to natural signals. Advances in Neural Information Processing Systems (2003): 277-284.
- ^ N. Brenner, S. P. Strong, R. Koberle, W. Bialek, and R. R. de Ruyter van Steveninck. "Synergy in a neural code. Neural Comp., 12:1531-1552, 2000.