Garriga et al. examined persistent changes in DNA methylation due to nerve injury, but their conclusions appear to be based on a flawed statistical analysis. The authors used genome-wide assays (DREAM and RRBS) which test thousands of individual genomic locations, however they did not take into consideration these multiple comparisons. Instead, they applied a nominal p-value threshold of 0.05. Taking the multiple comparisons into account would likely invalidate the reported effects, which can be explained in terms of the expected rate of false positives from such an analysis.
Garriga et al. tested significance of methylation differences between chronic pain and control animals at 32,688 separate sites in the genome, meaning one would expect roughly 1,600 nominally significant values with a threshold of p<0.05 even if there was no meaningful difference. Using this criterion, Garriga et al. found only ~7.6% of sites differentially methylated at 3 days and ~4.5% at 3 weeks post-surgery, consistent with the null hypothesis (Fig. 1A,B). Based on the reported p-values, all of which are >1E-6, the data do not appear to support a significant effect of nerve injury on DNA methylation in these samples.
To account for multiple comparisons, the standard method for statistical analysis of epigenome-wide data sets is false discovery rate (FDR) correction (Benjamini and Hochberg, 1995; Storey and Tibshirani, 2003). Careful attention to multiple comparisons is critical...
Show MoreGarriga et al. examined persistent changes in DNA methylation due to nerve injury, but their conclusions appear to be based on a flawed statistical analysis. The authors used genome-wide assays (DREAM and RRBS) which test thousands of individual genomic locations, however they did not take into consideration these multiple comparisons. Instead, they applied a nominal p-value threshold of 0.05. Taking the multiple comparisons into account would likely invalidate the reported effects, which can be explained in terms of the expected rate of false positives from such an analysis.
Garriga et al. tested significance of methylation differences between chronic pain and control animals at 32,688 separate sites in the genome, meaning one would expect roughly 1,600 nominally significant values with a threshold of p<0.05 even if there was no meaningful difference. Using this criterion, Garriga et al. found only ~7.6% of sites differentially methylated at 3 days and ~4.5% at 3 weeks post-surgery, consistent with the null hypothesis (Fig. 1A,B). Based on the reported p-values, all of which are >1E-6, the data do not appear to support a significant effect of nerve injury on DNA methylation in these samples.
To account for multiple comparisons, the standard method for statistical analysis of epigenome-wide data sets is false discovery rate (FDR) correction (Benjamini and Hochberg, 1995; Storey and Tibshirani, 2003). Careful attention to multiple comparisons is critical to avoid misinterpretation of large-scale neuronal epigenomics data.
References:
Benjamini Y, Hochberg Y (1995) Controlling the false discovery rate: a practical and powerful approach to multiple testing. J R Stat Soc Series B Stat Methodol:289–300.
Storey JD, Tibshirani R (2003) Statistical significance for genomewide studies. Proc Natl Acad Sci U S A 100:9440–9445.