On the process of comparing sentences against pictures

Abstract

The present study outlines a theory of how people compare sentences against pictures. This theory was tested in four experiments in which Ss were timed as they judged whether a sentence (e.g., Star isn't above plus) was true or false of a picture (e.g., ⁺_∗). The latencies in these tasks were consistent with the thesis that: (1) sentences are represented in terms of elementary propositions; (2) pictures are encoded in the same interpretive format; (3) these two codes are compared in an algorithmic series of mental operations, each of which contributes additively to the response latency; and (4) sentence encoding, picture encoding, comparing, and responding are four serially ordered stages, and their component latencies are additive. From these results, it was also possible to rule out certain explanations based on visual imagery, conversion (e.g., converting isn't above into is below), reading time, normative word frequencies, and other factors. Finally, it was shown that this theory is consistent with previous studies on sentence comprehension, sentence verification, concept verification, and other related phenomena.