Skip to main content

Main menu

  • HOME
  • CONTENT
    • Early Release
    • Featured
    • Current Issue
    • Issue Archive
    • Collections
    • Podcast
  • ALERTS
  • FOR AUTHORS
    • Information for Authors
    • Fees
    • Journal Clubs
    • eLetters
    • Submit
  • EDITORIAL BOARD
  • ABOUT
    • Overview
    • Advertise
    • For the Media
    • Rights and Permissions
    • Privacy Policy
    • Feedback
  • SUBSCRIBE

User menu

  • Log in
  • My Cart

Search

  • Advanced search
Journal of Neuroscience
  • Log in
  • My Cart
Journal of Neuroscience

Advanced Search

Submit a Manuscript
  • HOME
  • CONTENT
    • Early Release
    • Featured
    • Current Issue
    • Issue Archive
    • Collections
    • Podcast
  • ALERTS
  • FOR AUTHORS
    • Information for Authors
    • Fees
    • Journal Clubs
    • eLetters
    • Submit
  • EDITORIAL BOARD
  • ABOUT
    • Overview
    • Advertise
    • For the Media
    • Rights and Permissions
    • Privacy Policy
    • Feedback
  • SUBSCRIBE
PreviousNext
Behavioral/Systems/Cognitive

Synergy, Redundancy, and Independence in Population Codes, Revisited

Peter E. Latham and Sheila Nirenberg
Journal of Neuroscience 25 May 2005, 25 (21) 5195-5206; DOI: https://doi.org/10.1523/JNEUROSCI.5319-04.2005
Peter E. Latham
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Sheila Nirenberg
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
  • Article
  • Figures & Data
  • Info & Metrics
  • eLetters
  • PDF
Loading

Article Figures & Data

Figures

  •   Figure 1.
    • Download figure
    • Open in new tab
    • Download powerpoint
    Figure 1.

    Estimating conditional response distributions from data. a, Estimate of the correlated response distribution, p(r1, r2|s), for a single stimulus, s. The responses, r1 and r2, are taken to be spike count in a 300 ms window. They range from 0 to 19, so there are 400 (= 20 × 20) bins. A total of 250 trials were used, which leads to a very noisy estimate. b, Estimate of the independent response distribution, p(r1|s)p(r2|s). The single neuron distributions, p(r1|s) and p(r2|s), can be estimated individually, using all 250 trials for each one, and the joint distribution can then be constructed by multiplying them together. This leads to a much smoother (and more accurate) estimate of the probability distribution.

  •   Figure 2.
    • Download figure
    • Open in new tab
    • Download powerpoint
    Figure 2.

    Correlations can exist without being important for decoding. a, Correlated response distributions for four stimuli, shown as solid lines. For each stimulus, the responses lie along the line segments indicated by the arrows. [Formally, p(r1, r2|si) ∝ δ(si - (r2 - air1)), where ai is the slope of the line segment, and there is an implicit cutoff when r1 is below some minimum or above some maximum.] If the stimulus is known, r1 predicts r2 and vice versa, making the responses perfectly correlated. Because the responses form disjoint sets, all responses are uniquely decodable. b, Independent response distributions for the same four stimuli, shown as open boxes (the correlated distributions are shown also, as dashed lines). For each stimulus, the responses lie inside the boxes indicated by the arrows. The boxes overlap, and, if a response were to occur in the overlap region, it would not be uniquely decodable, because it could have been produced by more than one stimulus. However, the responses never land in this region (because they always land on the dashed lines). Thus, a decoder built with no knowledge of the correlational structure would be able to decode the true responses perfectly. c, A very similar set of correlated distributions. A decoder with knowledge of the correlations would be able to decode all responses perfectly. d, The independent response distributions derived from c. The true responses can lie in the overlap region, and a decoder that had no knowledge of the correlational structure would not be able to decode such responses perfectly. Thus, the correlations here are clearly important for decoding.

  •   Figure 3.
    • Download figure
    • Open in new tab
    • Download powerpoint
    Figure 3.

    ΔIshuffled can be both positive and negative when correlations are not important for decoding. a, Correlated response distributions for two stimuli, s1 and s2, which occur with equal probability. For each stimulus, the responses fall inside the boxes labeled by that stimulus. Because the responses are disjoint, all are uniquely decodable. b, Independent response distributions for the same stimuli. (The center boxes are offset so both can be seen.) Responses in the center box, which occur on one-quarter of the trials, provide no information about the stimulus. Thus, the independent responses provide less information than the true responses (because they are sometimes ambiguous), so ΔIshuffled > 0 (see Appendix A). However, as with Figure 2a, the true responses never land in the ambiguous region, so a decoder that has no knowledge of the correlations will decode exactly as well as one that has full knowledge of them. c, A different set of correlated response distributions, also for two stimuli. In this case, the responses land in the center box on one-half of the trials, and thus there is ambiguity about what the stimulus is on one-half the trials. d, The corresponding independent distribution (which is the same as in b). Here, the independent responses are ambiguous on only one-quarter of the trials, so they provide more information about the stimulus than the true responses. Thus, for this example, ΔIshuffled < 0 (see Appendix A). However, regardless of whether a decoder knows about the correlations, if a response lands in the overlap region, the stimulus probabilities are the same (one-half for each), so, as in a, knowledge of the correlations is not necessary for decoding.

  •   Figure 4.
    • Download figure
    • Open in new tab
    • Download powerpoint
    Figure 4.

    A synergistic code in which correlations are not important for decoding. a, Correlated response distributions for three stimuli, s1, s2, and s3, which occur with equal probability. For each stimulus, the responses fall inside the boxes. Because the responses form disjoint sets, all responses are uniquely decodable. For this distribution, it is not hard to show that Embedded Image (see Appendix A). b, Independent response distributions for the same stimuli. (The boxes along the diagonal are offset so both can be seen.) If a response were to land in a box along the diagonal, it would not be uniquely decodable; it could have been produced by two stimuli. However, as with Figures 2a and 3a, the responses never occur along the diagonal. Thus, even if a decoder knew nothing at all about the correlational structure, it would be able to decode perfectly all responses that actually occur (which are the only ones that matter). The probability distributions for this figure were derived from Schneidman et al. (2003); see Discussion.

  •   Figure 5.
    • Download figure
    • Open in new tab
    • Download powerpoint
    Figure 5.

    Highly synergistic codes are efficient but typically very difficult to decode. a, Two-neuron response distribution for five stimuli, color coded for clarity. The distribution is of the form p(s|r1, r2) ∝ δ(s - (r1 - r2)), with both r1 and r2 restricted to lie between 0 and 1. The stimulus, s, is a continuous variable that is uniformly distributed between -1 and 1. Observing both responses tells us exactly what the stimulus is, so the responses provide an infinite amount of information (the stimulus is specified with infinite precision). Observing any one response, however, provides only a finite amount of information about the stimulus. Consequently, ΔIsynergy = ∞. This coding scheme is advantageous because it can transmit an infinite amount of information and is easy to decode (s = r1 - r2). Note, however, that it requires perfect correlation, which is not biologically plausible. b, A distribution in which the r1-r2 plane was scrambled: it was divided into a 100 × 100 grid, and the squares in each column were randomly permuted. If we knew the scrambling algorithm, we could decode responses perfectly: Embedded Image, where Embedded Image is an operator that transforms r1 and r2 in scrambled coordinates to r1 - r2 in unscrambled ones. However, the decoder would have to store 100 log2(100!) bits just to unscramble the responses (log2(100!) bits per column), and, in general, for an n × n grid, it would have to store n log2(n!) ∼ n2log2(n/e) bits. Moreover, adding even a small amount of noise would destroy the ability of the responses to tell us anything about the stimulus. In the space of response distributions, non-smooth ones such as this are over-whelmingly more likely than the smooth one shown in a. Thus, minimizing redundancy (which leads to maximum synergy) would almost always produce an encoding scheme that is virtually impossible to decode.

Back to top

In this issue

The Journal of Neuroscience: 25 (21)
Journal of Neuroscience
Vol. 25, Issue 21
25 May 2005
  • Table of Contents
  • About the Cover
  • Index by author
Email

Thank you for sharing this Journal of Neuroscience article.

NOTE: We request your email address only to inform the recipient that it was you who recommended this article, and that it is not junk mail. We do not retain these email addresses.

Enter multiple addresses on separate lines or separate them with commas.
Synergy, Redundancy, and Independence in Population Codes, Revisited
(Your Name) has forwarded a page to you from Journal of Neuroscience
(Your Name) thought you would be interested in this article in Journal of Neuroscience.
CAPTCHA
This question is for testing whether or not you are a human visitor and to prevent automated spam submissions.
Print
View Full Page PDF
Citation Tools
Synergy, Redundancy, and Independence in Population Codes, Revisited
Peter E. Latham, Sheila Nirenberg
Journal of Neuroscience 25 May 2005, 25 (21) 5195-5206; DOI: 10.1523/JNEUROSCI.5319-04.2005

Citation Manager Formats

  • BibTeX
  • Bookends
  • EasyBib
  • EndNote (tagged)
  • EndNote 8 (xml)
  • Medlars
  • Mendeley
  • Papers
  • RefWorks Tagged
  • Ref Manager
  • RIS
  • Zotero
Respond to this article
Request Permissions
Share
Synergy, Redundancy, and Independence in Population Codes, Revisited
Peter E. Latham, Sheila Nirenberg
Journal of Neuroscience 25 May 2005, 25 (21) 5195-5206; DOI: 10.1523/JNEUROSCI.5319-04.2005
del.icio.us logo Digg logo Reddit logo Twitter logo Facebook logo Google logo Mendeley logo
  • Tweet Widget
  • Facebook Like
  • Google Plus One

Jump to section

  • Article
    • Abstract
    • Introduction
    • Results
    • Discussion
    • Appendix A: Calculation of ΔI, ΔIshuffled and ΔIsynergy
    • Appendix B: Information-theoretic cost of ignoring correlations
    • Footnotes
    • References
  • Figures & Data
  • Info & Metrics
  • eLetters
  • PDF

Responses to this article

Respond to this article

Jump to comment:

No eLetters have been published for this article.

Related Articles

Cited By...

More in this TOC Section

  • Episodic Reinstatement in the Medial Temporal Lobe
  • Musical Expertise Induces Audiovisual Integration of Abstract Congruency Rules
  • The Laminar Development of Direction Selectivity in Ferret Visual Cortex
Show more Behavioral/Systems/Cognitive
  • Home
  • Alerts
  • Visit Society for Neuroscience on Facebook
  • Follow Society for Neuroscience on Twitter
  • Follow Society for Neuroscience on LinkedIn
  • Visit Society for Neuroscience on Youtube
  • Follow our RSS feeds

Content

  • Early Release
  • Current Issue
  • Issue Archive
  • Collections

Information

  • For Authors
  • For Advertisers
  • For the Media
  • For Subscribers

About

  • About the Journal
  • Editorial Board
  • Privacy Policy
  • Contact
(JNeurosci logo)
(SfN logo)

Copyright © 2023 by the Society for Neuroscience.
JNeurosci Online ISSN: 1529-2401

The ideas and opinions expressed in JNeurosci do not necessarily reflect those of SfN or the JNeurosci Editorial Board. Publication of an advertisement or other product mention in JNeurosci should not be construed as an endorsement of the manufacturer’s claims. SfN does not assume any responsibility for any injury and/or damage to persons or property arising from or related to any use of any material contained in JNeurosci.