Skip to main content
Log in

The Principle of Inverse Effectiveness in Multisensory Integration: Some Statistical Considerations

Brain Topography Aims and scope Submit manuscript

Abstract

The principle of inverse effectiveness (PoIE) in multisensory integration states that, as the responsiveness to individual sensory stimuli decreases, the strength of multisensory integration increases. I discuss three potential problems in the analysis of multisensory data with regard to the PoIE. First, due to ‘regression towards the mean,’ the PoIE may often be observed in datasets that are analysed post-hoc (i.e., when sorting the data by the unisensory responses). The solution is to design discrete levels of stimulus intensity a priori. Second, due to neurophysiological or methodological constraints on responsiveness, the PoIE may be, in part, a consequence of ‘floor’ and ‘ceiling’ effects. The solution is to avoid analysing or interpreting data that are too close to the limits of responsiveness, enabling both enhancement and suppression to be reliably observed. Third, the choice of units of measurement may affect whether the PoIE is observed in a given dataset. Both relative (%) and absolute (raw) measurements have advantages, but the interpretation of both is affected by systematic changes in response variability with changes in response mean, an issue that may be addressed by using measures of discriminability or effect-size such as Cohen’s d. Most importantly, randomising or permuting a dataset to construct a null distribution of a test parameter may best indicate whether any observed inverse effectiveness specifically characterises multisensory integration. When these considerations are taken into account, the PoIE may disappear or even reverse in a given dataset. I conclude that caution should be exercised when interpreting data that appear to follow the PoIE.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3

Similar content being viewed by others

Notes

  1. Additional data and programs, a bibliography, numerous re-analyses of published datasets, and the results of numerical simulations concerning the principle of inverse effectiveness are or will shortly be freely available on the author's website, at http://www.neurobiography.info.

  2. To obtain estimates of effect-sizes, the SD for each response was estimated using the equation relating the mean and SD of responses from Fig. 2. The upper and lower 95% confidence limits for the slope and intercept of this equation were also used to estimate upper and lower confidence limits on the effect sizes (see Fig. 3d).

References

  • Alvarado JC, Stanford TR, Vaughan JW, Stein BE (2007a) Cortex mediates multisensory but not unisensory integration in superior colliculus. J Neurosci 27:12775–12786

    Article  PubMed  CAS  Google Scholar 

  • Alvarado JC, Vaughan JW, Stanford TR, Stein BE (2007b) Multisensory versus unisensory integration: contrasting modes in the superior colliculus. J Neurophysiol 97:3193–3205

    Article  PubMed  Google Scholar 

  • Beauchamp MS (2005) Statistical criteria in fMRI studies of multisensory integration. Neuroinformatics 3:93–113

    Article  PubMed  Google Scholar 

  • Diederich A, Colonius H (2008) When a high-intensity “distractor” is better then a low-intensity one: modeling the effect of an auditory or tactile nontarget stimulus on visual saccadic reaction time. Brain Res 1242:219–230

    Article  PubMed  CAS  Google Scholar 

  • Driver J, Noesselt T (2008) Multisensory interplay reveals crossmodal influences on ‘sensory-specific’ brain regions, neural responses, and judgments. Neuron 57:11–23

    Article  PubMed  CAS  Google Scholar 

  • Gilmeister H, Eimer M (2007) Tactile enhancement of auditory detection and perceived loudness. Brain Res 1160:58–68

    Article  Google Scholar 

  • Gondan M, Niederhaus B, Rösler F (2005) Multisensory processing in the redundant-target effect: a behavioral and event-related potential study. Percept Psychophys 67:713–726

    PubMed  Google Scholar 

  • Hecht D, Reiner M, Karni A (2008) Multisensory enhancement: gains in choice and in simple response times. Exp Brain Res 189:133–143

    Article  PubMed  Google Scholar 

  • Holmes NP (2007) The law of inverse effectiveness in neurons and behaviour: multisensory integration versus normal variability. Neuropsychologia 45:3340–3345

    Article  PubMed  Google Scholar 

  • Holmes NP, Calvert GA, Spence C (2004) Extending or projecting peripersonal space with tools? Multisensory interactions highlight only the distal and proximal ends of tools. Neurosci Lett 372:62–67

    Article  PubMed  CAS  Google Scholar 

  • Holmes NP, Calvert GA, Spence C (2007) Tool use changes multisensory interactions in seconds: evidence from the crossmodal congruency task. Exp Brain Res 183:465–476

    Article  PubMed  Google Scholar 

  • Kayser C, Petkov CI, Logothetis NK (2008) Visual modulation of neurons in auditory cortex. Cereb Cortex 18:1560–1574

    Article  PubMed  Google Scholar 

  • King AJ, Palmer AR (1985) Integration of visual and auditory information in bimodal neurones in the guinea-pig superior colliculus. Exp Brain Res 60:492–500

    Article  PubMed  CAS  Google Scholar 

  • Lakatos P, Chen C-M, O’Connell MN, Mills A, Schroeder CE (2007) Neuronal oscillations and multisensory interaction in primary auditory cortex. Neuron 53:279–292

    Article  PubMed  CAS  Google Scholar 

  • Laurienti PJ, Perrault TJ, Stanford TR, Wallace MT, Stein BE (2005) On the use of superadditivity as a metric for characterizing multisensory integration in functional neuroimaging studies. Exp Brain Res 166:289–297

    Article  PubMed  Google Scholar 

  • Longo MR, Cardozo S, Haggard P (2008) Visual enhancement of touch and the bodily self. Conscious Cogn 17:1181–1191

    Article  PubMed  Google Scholar 

  • Meredith MA, Stein BE (1983) Interactions among converging sensory inputs in the superior colliculus. Science 221:389–391

    Article  PubMed  CAS  Google Scholar 

  • Meredith MA, Stein BE (1986a) Spatial factors determine the activity of multisensory neurons in cat superior colliculus. Brain Res 365:350–354

    Article  PubMed  CAS  Google Scholar 

  • Meredith MA, Stein BE (1986b) Visual, auditory, and somatosensory convergence on cells in superior colliculus results in multisensory integration. J Neurophysiol 56:640–662

    PubMed  CAS  Google Scholar 

  • Meredith MA, Nemitz JW, Stein BE (1987) Determinants of multisensory integration in superior colliculus neurons. I. Temporal factors. J Neurosci 7:3215–3229

    PubMed  CAS  Google Scholar 

  • Perrault TJ Jr, Vaughan JW, Stein BE, Wallace MT (2003) Neuron-specific response characteristics predict the magnitude of multisensory integration. J Neurophysiol 90:4022–4026

    Article  PubMed  Google Scholar 

  • Perrault TJ Jr, Vaughan JW, Stein BE, Wallace MT (2005) Superior colliculus neurons use distinct operational modes in the integration of multisensory stimuli. J Neurophysiol 93:2575–2586

    Article  PubMed  Google Scholar 

  • Ross LA, Saint-Amour D, Leavitt VM, Javitt DC, Foxe JJ (2007) Do you see what I am saying? Exploring visual enhancement of speech comprehension in noisy environments. Cereb Cortex 17:1147–1153

    Article  PubMed  Google Scholar 

  • Serino A, Farnè A, Rinaldesi ML, Haggard P, Làdavas E (2007) Can vision of the body ameliorate impaired somatosensory function? Neuropsychologia 45:1101–1107

    Article  PubMed  Google Scholar 

  • Skaliora I, Doubell TP, Holmes NP, Nodal FR, King AJ (2004) Functional topography of converging visual and auditory inputs to neurons in the rat superior colliculus. J Neurophysiol 92:2933–2946

    Article  PubMed  Google Scholar 

  • Stevenson RA, James TW (2009) Audiovisual integration in human superior temporal sulcus: inverse effectiveness and the neural processing of speech and object recognition. NeuroImage 44:1210–1223

    Article  PubMed  Google Scholar 

  • Stevenson RA, Kim S, James T (in press) An additive-factors design to disambiguate neuronal and areal convergence: measuring multisensory interactions between audio, visual, and haptic sensory streams using fMRI. Exp Brain Res. doi:10.1007/s00221-009-1783-8

  • Sumby WH, Pollack I (1954) Visual contribution to speech intelligibility in noise. J Acoust Soc Am 26:212–215

    Article  Google Scholar 

Download references

Acknowledgements

NPH was supported by postdoctoral fellowships from the Interdisciplinary Center for Neural Computation and the Golda Meir foundation, Hebrew University of Jerusalem, Israel. Thanks to Tamar Makin, Ryan Stevenson, Eli Nelken, and the two anonymous reviewers for very helpful comments.

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Nicholas P. Holmes.

Additional information

This article is published as part of the Special Issue on Multisensory Integration.

Electronic Supplementary Material

Below is the link to the electronic supplementary material.

(DOC 61 kb)

Supplementary Fig. 1a

Inverse effectiveness in randomly-generated datasets when using a relative (%) measurement of multisensory integration. Multisensory datasets were generated randomly from a unisensory dataset of 500 points drawn from a Gaussian distribution (with a mean and SD of 0.5 and 0.15 respectively, x~N(0.5,0.15)), by adding one of five levels of Gaussian noise (i.e., Y = x+ae, where e~N(0,1), and a = 0, 0.025, 0.050, 0.075, 0.100). The resulting datasets were then rescaled to fit a range of (0≤Y≤1) (by subtracting the minimum value and dividing by the range). To generate different standard deviations of the multisensory dataset with respect to the unisensory dataset (and thereby the slope of the relationship between multisensory and unisensory data), the multisensory data were multiplied by one of seven constants (0.7, 0.8, 0.9, 1.0, 1.1, 1.2, 1.3, plotted as separate panels across the horizontal axis of the figure). To generate different means of the multisensory dataset with respect to the unisensory dataset (and thereby the intercept of the relationship between multisensory and unisensory data), one of seven constants was added to the multisensory data (-0.15, -0.10, -0.05, 0, 0.05, 0.10, 0.15, plotted as separate panels up the vertical axis of the figure). The five resulting multisensory datasets in each of the 49 separate simulations were highly correlated with the unisensory datasets (r≈1, 0.99, 0.95, 0.90, 0.83, plot as differently-coloured data points in each panel, respectively dark magenta, red, green, cyan, and blue). The data were converted into a relative (%) index of multisensory interaction (MSI=100*[Multisensory response– Unisensory response]/Unisensory response), separately for multisensory enhancements (MSI≥0, shown on a logarithmic scale in the upper sub-panel of each plot)) and suppressions (MSI<0, on a linear scale in the lower sub-panel of each data plot). For each dataset, multisensory integration is plot on the y-axis, and the maximum unisensory response on the x-axis. A power regression (enhancement) and logarithmic regression (suppression) lines and associated r2 values are also shown. Linear regression on the suppression data showed the same trends, but tended to provide worse fits (lower r2 values) to the data. In general, as the correlation between unisensory and multisensory datasets decreases (from magenta to red, green, cyan, and blue data points), the slope and intercept of the regression of multisensory integration on the maximum unisensory response increases. The data indicate that inverse effectiveness is a common feature of datasets with a linear relationship between unisensory and multisensory responses, but that as the correlation between these variables decreases, inverse effectiveness increases. Critical 1-tailed R2 value = 0.005 (p<.05). The raw data and other files used to produce these figures are freely available for download from the author's website. (TIFF 1669 kb)

Supplementary Fig. 1b

As for Supplementary Figure 1A, except that the index of multisensory integration was calculated as an absolute (raw) difference (MSI=Multisensory response – Unisensory response). All data are plot on linear axes, and linear regressions are shown. Inverse effectiveness is less commonly observed for absolute (raw) than for the relative (%) measurements, but the effects of increasing variability and decreasing correlation between unisensory and multisensory data are the same – as the correlation decreases, inverse effectiveness increases. In contrast to the relative (%) measurements, multisensory suppression with absolute (raw) data generally shows negative, not positive slopes. Critical 1-tailed R2 value = 0.005 (p<.05). (TIFF 1388 kb)

Supplementary Fig. 2

Six commonly-used equations constrain the possible values that multisensory integration can take, given absolute floor (0.0) and ceiling (1.0) effects. max & min: maximum & minimum possible multisensory integration, given ceiling and floor effects. a) Multisensory interaction, MSI=100%*[(Multisensory-Maximum Unisensory)/Maximum Unisensory], e.g., Meredith and Stein 1983; b) Multisensory interaction, MSI=100%*[(Multisensory-Sum of Unisensory)/(Sum of Unisensory)], e.g., King and Palmer 1985; c) Multisensory contrast, MSC=100%*[(Multisensory-Maximum Unisensory)/(Multisensory+Maximum Unisensory)], e.g., Avillac et al. 2007; d) Multisensory additivity MSA=100%*[(Multisensory-Sum of Unisensory)/(Multisensory+Sum of Unisensory)], e.g., Avillac et al. 2007; e) Multisensory gain, MSG=100%*[(Multisensory-Maximum Unisensory)/Multisensory], e.g., Ross et al. 2007); f) Multisensory gain relative to ceiling effects, MS(r)G=100*[(Multisensory-Maximum Unisensory)/(Ceiling-Maximum Unisensory)], e.g., Ross et al. 2007; Sumby and Pollack 1954. (TIFF 153 kb)

Supplementary Fig. 3A

Behavioural data from 180 healthy human participants performing 12 conditions of a multisensory spatial congruency task (Holmes et al. 2004, 2007). Participants responded as quickly as possible using one of two foot pedals to the elevation (upper vs. lower) of vibrotactile target stimuli presented on either thumb or forefinger, while trying to ignore the elevation (upper vs. lower, i.e., congruent or incongruent with the vibrotactile target elevation) of simultaneous visual distractor stimuli, presented near to or far from the vibrotactile targets. The variability (SD) of RTs for all 12 conditions (total n = 2160) is plot against the mean RT. a) The SD changes systematically with the mean. b) Plotting the coefficient of variation (SD/mean) results in 'inverse effectiveness' – RTs are relatively more variable when the mean RT is longer (i.e., when 'responsiveness' is lower and participants perform worse on the task). Critical 1-tailed R2 value ≈ 0.001 (p<.05). (TIFF 567 kb)

Supplementary Fig. 3B

Electrophysiological data from whole-cell patch clamp recordings made from neurons in the intermediate and deep (multisensory) layers of the neonatal rat superior colliculus. Excitatory inputs were simulated by microstimulation of the superficial layers of the superior colliculus ('visual' inputs), and the nucleus of the brachium of the inferior colliculus ('auditory' inputs). Monosynaptic excitatory post-synaptic potentials (EPSPs) were recorded from putative multisensory cells in the intermediate and deeper layers of the superior colliculus. Sixteen responses where both the mean and SD of EPSP amplitude were available, are presented. a) The variability (SD) of EPSP amplitude increases (marginally significantly) with the mean. b) The coefficient of variation (SD/mean) of EPSP amplitude decreases significantly with the mean, showing inverse effectiveness – greater variability when the responses are weaker. Critical 1-tailed R2 value = 0.181 (p<.05). (TIFF 363 kb)

Supplementary Fig. 4A

Inverse effectiveness in behavioural choice RT data depends on how multisensory integration is calculated. The difference in RT between incongruent and congruent trials (the crossmodal congruency effect) is plot against the mean RT, separately for stimuli presented on the same-hand (same-location) and on different-hands (different locations), for 180 healthy human participants. Dark coloured symbols and regression lines, and the upper regression equation show trials with bimodal stimuli in the same location. Light coloured symbols and lines and the lower regression equation show trials with bimodal stimuli in different locations. a) Relative (%) measures show significant inverse effectiveness for same-location, and a significant reverse trend for different-location bimodal stimuli (congruency effect (%) = 100%*[(Multisensory congruency effect-Mean RT)/Mean RT]. b) Absolute (ms) measures show significant inverse effectiveness for same-location bimodal stimuli, but no significant trend for different-location stimuli (congruency effect (ms) = Multisensory congruency effect). c) Expressed as a percentage of the maximum possible change (i.e., the congruency effect relative to the imposed ceiling of 1500ms), the data show significant inverse effectiveness for same-location, but no trend for different-location stimuli (congruency effect (% of max) = 100%*[congruency effect/(1500-congruent RT)]. d) Effect-size measurements (the data shown in panel b, divided by the pooled SD of congruent and incongruent trials) show no significant trend for same-location, and a significant reverse trend for different-location stimuli. In summary, effect size measurements, which take into account the systematic changes of response variability with response mean show no inverse effectiveness. Other measurements, which do not take changes in variability into account, all show significant inverse effectiveness for same-location bimodal stimuli. Critical 1-tailed R2 value = 0.016 (p<.05). (TIFF 544 kb)

Supplementary Fig. 4B

Inverse effectiveness in whole cell patch-clamp recordings of EPSPs in cells from intermediate layers of the superior colliculus depends on how multisensory interactions are calculated. 12 cells showing multisensory enhancement (Multisensory response > Maximum unisensory response) are shown. a) Relative (%) enhancement (MSI = 100*[(Multisensory-Maximum Unisensory)/Maximum Unisensory]). b) Absolute (raw) enhancement (MSI (mV) = [Multisensory – Maximum Unisensory]. c) Enhancement relative to the maximum possible change (MSI (% of max) = 100%*[(Multisensory – Maximum unisensory)/(Maximum response across all cells and conditions – Maximum unisensory)]). d) Multisensory effect size = (MSI (d) = [(Multisensory-Maximum unisensory)/estimated pooled SD]. Only the relative percentage measurement showed significant inverse effectiveness. The meaning of this significant inverse effectiveness (if it is not simply artefactual) in these data is hard to interpret – these data were recorded from slices of brain of neonatal animals (12-15 days old) which had not yet opened their eyes, and from cells which could not specifically be identified as responding to naturalistic multisensory stimuli (see Wallace and Stein 1997, for the typical time-course of development of multisensory integration in postnatal cats). Critical 1-tailed R2 value = 0.247 (p<.05). (TIFF 321 kb)

Supplementary Fig. 5A

Randomisation of the data shown in Figure 3 illustrates the statistical dependence between measurements of multisensory interactions and the maximal unisensory response. Three randomisation procedures were run: Randomisation 1 (left column): The relationship between unisensory and multisensory responses was randomised – i.e., the cell labels for the unisensory data were randomised. Randomisation 2 (middle column): The unisensory and multisensory data were randomly resampled with replacement separately from the observed unisensory and multisensory distributions. Randomisation 3 (right column): The unisensory and multisensory data were randomly resampled with replacement from the combined unisensory and multisensory distributions. In each randomisation procedure, 1178 data points were randomised or sampled for unisensory and multisensory responses. Multisensory integration was calculated in one of 4 ways: a) Relative % MSI = 100*[(Multisensory-Unisensory)/Unisensory]; b) Absolute (raw) MSI = [Multisensory – Unisensory]; c) Relative (% of max) MSI = 100%*[(Multisensory – Unisensory)/(Maximum observed response – Unisensory)]; d) Effect-size (d) MSI = [(Multisensory – Unisensory)/estimated pooled SD]. The SD of each mean response was estimated using the best-fit equation from Figure 2: SD = 0.416mean + 1.27. The multisensory enhancements were identified (MSI>0), the linear and log slopes and correlations of multisensory enhancement on the unisensory response was calculated. The randomisation was repeated 10000 times, and the resulting distributions of the log slopes are shown in the each figure panel. Above each distribution, the upward-pointing black triangles show the lower (5%) and upper (95%) limits for the tails of the distributions. The downward-pointing green triangles show the observed log slopes for each measurement unit. For all randomisation procedures, and all units of measurement, the observed slope is substantially (and highly significantly) less negative or more positive (i.e., less inversely-effective) than the null distribution of slopes. The raw data and program scripts used to perform these randomisations are freely available for download from the author's website. (TIFF 340 kb)

Supplementary Fig. 5B

The procedures outlined in Supplementary Figure 5A were applied to the RT dataset shown in Supplementary Figure 4A (same-location data only, n=180). All the randomisation procedures and all the measurement units resulted in distributions of positive slopes (i.e., in the direction predicted by inverse effectiveness), but the obtained slopes (green triangles) were substantially (and highly significantly) less positive (i.e., less inversely-effective) than the null distributions. (TIFF 366 kb)

Supplementary Fig. 5C

The procedures outlined in Supplementary Figure 5A were applied to the EPSP dataset (n=14) shown in Supplementary Figure 4B. Since two unisensory responses were available, the maximum unisensory response in each randomisation was compared to the multisensory response. All the randomisation procedures and all the measurement units resulted in distributions of slopes shifted towards negative values (i.e., in the direction predicted by inverse effectiveness), but the obtained slopes (green triangles) were all less negative or more positive (i.e., less inversely-effective) than the mean of the null distributions. For Randomisation 1, two of the four obtained slopes (a: relative (%), and b: absolute (raw) measurements) were less inversely effective than the 95% cut-off of the null distributions, and 2 were within the cut-off. For Randomisation 2, only the relative (%) measurement was outside the 95% cut-off. For Randomisation 3, none of the four measures resulted in a slope different from the null distributions in either direction. (TIFF 379 kb)

Supplementary Fig. 6A

Binned randomisation of the data shown in Figure 3. In order to preserve the correlation between raw unisensory and multisensory datasets, the three randomisation procedures detailed in Supplementary Figure 5A were repeated, each time with a different bin width. The raw data were first sorted by the unisensory response. Randomisation or resampling was then performed only within restricted bins of unisensory responses. 10,000 randomisations were performed, with each of the 26 bin widths: 2, 3, 4, 5, 6, 8, 10, 12, 15, 19, 24, 31, 34, 38, 49, 62, 79, 98, 118, 147, 196, 236, 295, 393, 589, & 1178 responses. Each figure panel shows the relationship between two parameters of the resulting distributions: The correlation between raw unisensory and multisensory responses (x-axis), and the resulting slope of multisensory integration on unisensory responsiveness (y-axis). The horizontal axis of each ellipse covers 90% of the resulting correlations. The vertical axis of each ellipse covers 90% of the resulting slopes. The red dot and circle show the actual values obtained from the reconstructed dataset. The panels a-d and the three randomisation procedures are as shown in Supplementary Figure 5A. The raw data and program scripts used to perform these randomisations are freely available for download from the author's website. (TIFF 867 kb)

Supplementary Fig. 6B

Binned randomisation of the data shown in Supplementary Figure 4A. In order to preserve the correlation between raw unisensory and multisensory datasets, the three randomisation procedures detailed in Supplementary Figure 5B were repeated, each time with a different bin width. The raw data were first sorted by the unisensory response. Randomisation or resampling was then performed only within restricted bins of unisensory responses. 10,000 randomisations were performed, with each of the 25 bin widths: 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 18, 20, 23, 26, 30, 36, 45, 60, 90, & 180 responses. Each panel shows the relationship between two parameters of the resulting distributions: the correlation between raw unisensory and multisensory responses (x-axis), and the resulting slope of multisensory integration on unisensory responsiveness (y-axis). The horizontal axis of each ellipse covers 90% of the resulting correlations. The vertical axis of each ellipse covers 90% of the resulting slopes. The red dot and circle show the values obtained from the real dataset. The panels a-d and the three randomisation procedures are as shown in Supplementary Figure 5B. The raw data and program scripts used to perform these randomisations are freely available for download from the author's website. (TIFF 1232 kb)

Rights and permissions

Reprints and permissions

About this article

Cite this article

Holmes, N.P. The Principle of Inverse Effectiveness in Multisensory Integration: Some Statistical Considerations. Brain Topogr 21, 168–176 (2009). https://doi.org/10.1007/s10548-009-0097-2

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10548-009-0097-2

Keywords

Navigation