We used fMRI to investigate how moment-to-moment neural activity contributes to success or failure on individual trials of a visual working memory (WM) task. We found that different nodes of a distributed cortical network were activated to a greater extent for correct compared to incorrect trials during stimulus encoding, memory maintenance during delays, and at test. A logistic regression analysis revealed that the fMRI signal amplitude during the delay interval in a network of frontoparietal regions predicted successful performance on a trial-by-trial basis. Differential delay activity occurred even for only those trials in which BOLD activity during encoding was strong, demonstrating that it was not a simple consequence of effective versus ineffective encoding. Our results indicate that accurate memory depends on strong sustained signals that span the delay interval of WM tasks.