Nasal Respiration Entrains Human Limbic Oscillations and Modulates Cognitive Function

Zelano et al. (2016) reported multiple experimental findings to support their conclusion that nasal respiration entrains limbic oscillations and modulates human cognitive functioning. The presented data suggests, among other things, that breathing in/out affects memory retrieval more (d=0.86) than elaborate strategies designed to improve recall (d=0.49; Nairne et al., 2008). This claim is remarkable, if true. Our surprise at this finding led us to ask how likely it is that one would get such results with replication experiments using the same designs and sample sizes. Our analysis indicates that even when assuming the effects are real and accurately measured by the original experiments, the probability of success across three studies like those in Zelano et al. is estimated to be only 0.003. Given such low odds of replication success, we caution readers to be skeptical about the reported results and conclusions in Zelano et al.

For each behavioral experiment, we estimated the probability that a replication study would produce the same degree of success as the original study. Using the reported test statistics and Figures 8b, 9b, and 10a, we derived the sample means, standard deviations, and correlations. These sample statistics were then used as population estimates in 100,000 simulated experiments with the same sample sizes and statistical analyses as in Zelano et al. (2016). The proportion of simulated samples that generated the patterns of significance reported by Zelano et al. as support for their conclusions was: 0.08, 0.22, and 0.20 for experiments 1, 2, and 3, respectively. The estimated probability of all three experiments being successful is the product of these proportions. The R simulation code and related files are at https://osf.io/hdvy9/

These success proportions are low because Zelano et al. (2016) based their conclusions and interpretations on many different statistical outcomes. All the reported tests were successful, but often with results near the significance criterion (e.g., p=0.04), and a replication study with the same sample size is estimated to produce a significant outcome for a given hypothesis test only a bit more than half the time, simply due to the inherent variability in random sampling. The odds of the full set (for example, 8 tests in Experiment 1) being uniformly successful for any given random sample are very low. Furthermore, Zelano et al. (2016) reported several additional tests that all supported their conclusions, but their manuscript does not provide enough information to include them in our simulations. Moreover, we did not include the reported neurophysiological studies because the analyses were too complicated for our simulations. The probability of these additional outcomes also being successful must be even lower than our above estimate.

The excess success in Zelano et al. (2016) undermines their theoretical conclusions. Without knowing the details of the investigations, we cannot speculate on how their experiments produced their reported outcomes. The field of psychology has recently realized that some methods of sampling, analyzing, interpreting, and reporting empirical findings (e.g., Simmons et al., 2011; Francis, 2014) can inadvertently cause this kind of problem, and similar concerns may apply here. Whatever the cause, the findings in Zelano et al. (2016) do not provide appropriate empirical support for their conclusions.

References
Francis G (2014) The frequency of excess success for articles in Psychological Science. Psychonomic Bulletin & Review 21:1180-1187.

Nairne JS, Pandeirada JNS, Thompson SR (2008) Adaptive memory: The comparative value of survival processing. Psychological Science 19:176-180.

Simmons JP, Nelson LD, Simonsohn U (2011) False-positive psychology: Undisclosed flexibility in data collection and analysis allows presenting anything as significant. Psychological Science 22:1359-1366.