Stress is a potent modulator of learning and memory processes. Although there have been a few attempts in the literature to explain the diversity of effects (including facilitating, impairing, and lack of effects) described for the impact of stress on memory function according to single classification criterion, they have proved insufficient to explain the whole complexity of effects. Here, we review the literature in the field of stress and memory interactions according to five selected classifying factors (source of stress, stressor duration, stressor intensity, stressor timing with regard to memory phase, and learning type) in an attempt to develop an integrative model to understand how stress affects memory function. Summarizing on those conditions in which there was enough information, we conclude that high stress levels, whether intrinsic (triggered by the cognitive challenge) or extrinsic (induced by conditions completely unrelated to the cognitive task), tend to facilitate Pavlovian conditioning (in a linear-asymptotic manner), while being deleterious for spatial/explicit information processing (which with regard to intrinsic stress levels follows an inverted U-shape effect). Moreover, after reviewing the literature, we conclude that all selected factors are essential to develop an integrative model that defines the outcome of stress effects in memory processes. In parallel, we provide a brief review of the main neurobiological mechanisms proposed to account for the different effects of stress in memory function. Glucocorticoids were found as a common mediating mechanism for both the facilitating and impairing actions of stress in different memory processes and phases. Among the brain regions implicated, the hippocampus, amygdala, and prefrontal cortex were highlighted as critical for the mediation of stress effects.