We present a model for recovering the direction of heading of an observer who is moving relative to a scene that may contain self-moving objects. The model builds upon an algorithm proposed by Rieger and Lawton, based on earlier work by Longuet-Higgins and Prazdny. The algorithm uses velocity differences computed in regions of high depth variation to locate the focus of expansion, which indicates the observer's heading direction. We relate the behavior of the model to psychophysical observations regarding the ability of human observers to judge heading direction, and show how the model copes with self-moving objects in the environment.