Type I error rates and power analyses for single-point sensitivity measures

Percept Psychophys. 2008 Feb;70(2):389-401. doi: 10.3758/pp.70.2.389.

Abstract

Experiments often produce a hit rate and a false alarm rate in each of two conditions. These response rates are summarized into a single-point sensitivity measure such as d', and t tests are conducted to test for experimental effects. Using large-scale Monte Carlo simulations, we evaluate the Type I error rates and power that result from four commonly used single-point measures: d', A', percent correct, and gamma. We also test a newly proposed measure called gammaC. For all measures, we consider several ways of handling cases in which false alarm rate = 0 or hit rate = 1. The results of our simulations indicate that power is similar for these measures but that the Type I error rates are often unacceptably high. Type I errors are minimized when the selected sensitivity measure is theoretically appropriate for the data.

Publication types

  • Research Support, N.I.H., Extramural
  • Research Support, Non-U.S. Gov't

MeSH terms

  • Attention*
  • Bias
  • Cues*
  • Data Interpretation, Statistical*
  • Decision Making
  • Discrimination Learning*
  • Humans
  • Models, Statistical
  • Monte Carlo Method
  • Normal Distribution
  • Orientation
  • Pattern Recognition, Visual
  • Psychology, Experimental / statistics & numerical data*
  • Psychophysics
  • ROC Curve
  • Research Design / statistics & numerical data*
  • Sensitivity and Specificity
  • Signal Detection, Psychological*