Animals including humans experience objects in a certain environment, that is, a context. Same objects may have to be treated differently, or different objects may need to be treated similarly depending on contexts. Flexible behavioral choice in such ambiguous situations involves dynamic interactions among brain regions, but underlying neural mechanisms are poorly understood. In this article, prior studies that have examined (mostly in rodents) some of the brain regions involved in contextual processing of object information using goal-directed tasks are selectively reviewed. The current review identifies the hippocampus, prefrontal cortex (PFC) and perirhinal cortex (PER) as key regions for associating the same objects with different reward values and responses depending on the background visual context. The hippocampus is particularly important for contextual choice behavior when the context must be used as a conditional cue that can disambiguate reward-related 'meanings' of objects. The PER appears to play significant roles in such tasks during initial learning (but not so much for retrieval) because perturbations in the PER produce severe deficits in the acquisition of the contextual object memory task. Perturbations in the PFC also affect performance when flexible contextual responses should be made toward otherwise ambiguous objects.