A Theory of Geometric Constraints on Neural Activity for Natural Three-Dimensional Movement

Cosine directional tuning and multiplicatively linear speed modulation

The tuning rule in Equation 2 captures two main effects: cosine directional tuning and multiplicatively linear speed modulation, as clearly seen in its equivalent form: Equation 12where v = ‖v‖ is the reaching speed, the proportional factor p = ‖p‖ is length of the preferred direction vector p, and α is the angle between the instantaneous reaching direction and the preferred direction. Because the hand trajectory is approximately straight in normal reaching, the instantaneous velocity v is a vector that points in the same direction as the reaching direction. If a tuning function is cosine in three-dimensional space, it must also be cosine in any two-dimensional subspace, as in the examples in Figure 2.

A cosine function is a good approximation to the directional tuning data, although a circular normal function (Eq. EA17), with one more free parameter, tends to fit the data slightly better (Fig. 2). The residual can be roughly accounted for by an additional Fourier term, cos 2α, with an amplitude less than ∼10% of that of the original term, cos α (see further discussion in Appendix ).

The speed modulation effect predicted by Equation 12 is multiplicative; that is, the firing rate should be higher for faster reaching speed without affecting the shape of the cosine tuning function. This is approximately true as shown by Moran and Schwartz (1999), who, however, used the square root of firing rate in analysis so that the linearity of speed modulation on raw firing rate was not directly quantified. Indirect evidence for linear speed modulation includes trajectory reconstruction and the curvature power law (see below).

Neuronal population vector

Suppose the firing rate of each neuron i in a population (i = 1, 2, … , N) follows the same tuning rule as considered above: Equation 13The population vector u is defined as the vector sum of the preferred directions p_i weighted by firing rates relative to baselines (Georgopoulos et al., 1986): Equation 14where in the second step, Equation 13 is used. For the population vector u to be proportional to the true velocityv, namely: Equation 15the necessary and sufficient condition is that the preferred directions satisfy: Equation 16where λ is an arbitrary constant, I is 3 × 3 identity matrix, each p_i is a column vector, and p_i^T is a row vector (Mussa-Ivaldi, 1988; Gaál, 1993; Salinas and Abbott, 1994;Sanger, 1994). In particular, when p_i are distributed uniformly, as is roughly true for cells in motor cortex (Georgopoulos et al., 1988), the condition in Equation 16 is satisfied so that Equation 15 follows as a consequence. Then the population vector approximates the reaching direction and reaching velocity (Moran and Schwartz, 1999).

Trajectory reconstruction

One implication of Equation 15 is that integration over the population over time can reconstruct the hand trajectory, up to a scaling constant: Equation 17where r(t) is hand position at time t. This is consistent with the finding that adding up the population vector head-to-tail approximately reproduced the shape of the hand trajectory (Schwartz, 1993, 1994), because head-to-tail addition is a discrete approximation to the continuous vector integration.

Curvature power law

While drawing, the hand moves more slowly when the trajectory is more highly curved, and obeys a power law: Equation 18where ω is instantaneous angular velocity with respect to an instantaneous center determined by the local curvature κ of the trajectory, and B is a constant (Lacquaniti et al., 1983).Schwartz (1994) showed that the changing direction of the population vector of cells in motor cortex of monkeys followed the same power law during drawing. This is consistent with Equation 15, which requires that the population vector u be proportional to the instantaneous hand velocity v, up to a possible time difference. This form of the power law involves only the direction of population vector u. To test its length u = ‖u‖ or the linearity of firing rate modulation by reaching speed, one may use the equivalent form of the power law: Equation 19where v = rω is the hand speed and r = 1/κ is the local radius for the curvature of the trajectory. The length of the population vector u is proportional to the hand speed v if and only if the population vector follows the same power law in Equation 19.

Reaching distance

Fu et al. (1993) reported a nearly linear correlation between firing rates of cells in motor cortex and reaching distance. Although this result was somewhat confounded by faster reaching for longer distances, it raises the question of the general effect of reaching distance. A linear distance effect would be consistent with the basic model in Equation 2, which implies that: Equation 20where vector d =r(T) − r(0) is the final displacement from the starting position r(0), assuming the preferred direction p is approximately constant along movement trajectory. The dot product p · dimplies a linear relation between the reaching distance and the total spike count above baseline, together with a cosine directional tuning, regardless of the exact time course of hand velocity.

Note that in Equation 20 the baseline rate f₀has been subtracted. Because the baseline rate itself may contain a linear speed component as in Equation 9 (Moran and Schwartz, 1999), its contribution to total spike count should be: Equation 21where, for simplicity, a₁ = a₂ = a₃ = 0 has been assumed to ignore the effect of static hand position. Because the last term is proportional to the reaching distance ‖d‖ but independent of the reaching direction, it might account for the observation that the modulation of overall firing rates by reaching distance was often linear but insensitive to the reaching direction (Fu et al., 1993; Turner and Anderson, 1997).

Curl-free distribution of preferred direction

Caminiti et al. (1990, 1991) reported that the preferred direction of a motor cortical neuron often varied with the starting point of hand movement. This is allowed by the gradient theory, provided that this vector field is curl-free, according to Equation 5 or 6. A constant preferred direction field is always allowed because it has zero curl. The curl-free condition constrains how the preferred direction of a neuron may vary in different parts of space. For example, it rules out the possibility of any circular arrangement of the preferred directions, such as that in the two-joint planar arm example shown in Figure 3. The existing data do not include enough points to compute the curl (see Appendix ). Further experiments would be needed to test whether the prediction of the gradient theory is correct.

Elbow position

Scott and Kalaska (1997) found that the preferred directions of some motor cortical cells were altered when the monkey had to reach unnaturally with the elbow raised to shoulder level. In the current theoretical framework, adding elbow position as a free parameter is equivalent to adding one rotation variable ϕ, for example, the angle between the horizontal plane and the plane determined by the hand, elbow, and shoulder. The same theoretical argument yields the tuning rule: Equation 22where K is a coefficient that may depend on both hand position and elbow position. This formula implies two new effects. The first is that now the preferred direction vector, both its direction and length, may depend on the elbow position ϕ as well as the hand position (x, y, z): Equation 23as reported by Scott and Kalaska (1997). The second effect, a new prediction, is that the firing rate may contain a component proportional to the angular speed dϕ/dt of elbow rotation.

How does this case relate to our earlier results with hand position as the only free parameter? In the preceding sections, reaching was assumed to be “stereotypical” in the sense that the elbow position can be determined completely by the hand position, ignoring forearm rotation. This assumption may not be true if the final posture sometimes depends also on the initial hand position (Soechting et al., 1995). However, when comparing reaching movements starting from the same initial hand position, it is reasonable to assume that for stereotypical reaching, the elbow angle ϕ can be completely determined by the hand position (x, y, z), or ϕ = ϕ(x, y, z). Then the time derivative of ϕ, after expanding by the chain rule, can be absorbed into the termp · v, yielding the original basic tuning rule in Equation 2. In other words, the assumption of stereotypical movement reduces the total degrees of freedom to 3, eliminating the elbow position as an independent variable. Although the elbow angle can still be used as a free parameter, it is no longer independent of the hand position. Only three parameters are independent in this case, and their exact choice does not affect the general form of the tuning rule (see Appendix for more discussion on coordinate-system independence).

Summary and discussion of more complex cases

As shown above, the basic tuning theory can naturally account for several important experimental results without making any specific assumptions about the exact variables encoded or details of the encoding. These results are generic properties independent of the exact functional interpretations. This generality makes sense because during stereotypical movement, redundant variables are inevitably constrained by the geometry and become highly correlated, so that they are likely to show similar tuning properties of the same general type. The theory presented here has formalized this intuition.

The relationship between cosine tuning properties and geometric constraints is also apparent in the studies of muscle activities and actions during reaching and isometric tasks. Basic properties resembling those for motor cortical cells have been reported, including approximately cosine directional tuning curves (but often with a small secondary peak opposite the preferred direction), speed sensitivity, and posture dependence (Flanders and Soechting, 1990; Flanders and Herrmann, 1992; Buneo et al., 1997).

The basic theory needs to be generalized in situations where the hand position is not the only free parameter. For example, force is one variable that is often correlated with the activity of motor cortex; recent examples related to directional tuning include tasks with static load (Kalaska et al., 1989) and varying isometric forces (Georgopoulos et al., 1992; Sergio and Kalaska, 1997).

As another example, preparatory activity in motor cortex before onset of movement can reflect the upcoming reaching direction, as is especially evident during instructed delay (Georgopoulos et al., 1989a), and can change rapidly in tasks requiring mental rotation (Georgopoulos et al., 1989b) or target switching (Pellizzer et al., 1995).

Moreover, when sensory and motor components were decoupled, some neurons even from primary motor cortex were more closely related to the visual movement of a cursor on the computer screen than to the joystick position or hand movements, in both one-dimensional (Alexander and Crutcher, 1990) and two-dimensional tasks (Shen and Alexander, 1997a). By contrast, in virtual reality experiments with visual distortion, motor cortical activity mainly followed the actual limb trajectory rather than the animal’s visual perception (Moran et al., 1995).

In addition, some differences exist among the neural activity from different brain areas, although they all show approximate cosine directional tuning (compare Fig. 2). For instance, compared with neurons in the motor cortex in a reaching task, the preferred directions in the cerebellum are more variable in repeated trials (Fortier et al., 1989), neurons in the parietal cortex are less sensitive to static load (Kalaska et al., 1990), and neurons in the premotor cortex are activated earlier, more transiently (Caminiti et al., 1991; Crammond and Kalaska, 1996), and affected more frequently by visual cues (Wise et al., 1992; Shen and Alexander, 1997b). In the motor cortex and elsewhere, there also exist neurons with complex properties that are either not task-related or hard to describe but still could have useful functions in a distributed network (Fetz, 1992;Zipser, 1992; Moody et al., 1998).

In most of these cases, there are additional free variables besides hand position. The linear theory may still yield useful results in these more complex cases after including these additional variables. For example, the planned movement direction is an independent variable, which could be used to describe some preparatory activity before overt hand movement. These new variables should be included when deriving the tuning rule, as demonstrated in the preceding section by adding the elbow position as a free variable in abducted reaching.

Cosine tuning and multiplicative speed modulation

The basic tuning rule in Equation 30 can be rewritten in its equivalent form: Equation 36where v = ‖v‖ is the speed of translation, ω = ‖ω‖ is the angular speed of rotation, p = ‖p‖ is the length of the preferred direction vector, q = ‖q‖ is the length of the preferred rotation vector, α is the angle between vectors p and v, and β is the angle between vectors q and ω.

In other words, given the particular view of a particular object, the response above baseline should be the sum of two components, one translational and one rotational. The translational component is proportional to the cosine of the angle between a fixed preferred translational direction and the actual translational direction. In addition, it is also modulated linearly by the speed of translation, which does not alter the shape of the tuning curve. Similarly, the rotational component is proportional to the cosine of the angle between a fixed preferred rotation axis and the actual rotation axis. In addition, the rotational component is also modulated linearly by the angular speed of rotation.

Distribution of preferred direction and preferred axis

Thus far, the view of the given object is assumed to be fixed. That is, the cosine tunings for both translation and rotation are defined with respect to a particular view of the object. When the view of the object changes, the preferred translational directionp and preferred rotation axis q of a motion-sensitive neuron may also change.

The theory constrains this change because the preferred translational direction p and the transformed preferred rotation axisq* are derived as gradient fields in Equations 31 and 33. Here the intermediate vector q* is related to the preferred rotation axis q in physical space by: Equation 37according to Equation 32. In three-dimensional space, where curl is defined, the gradient field implies that any three variables taken from the six variables (x, y, z, θ₁, θ₂, θ₃) must be curl-free. For example, when the position (x, y, z) of the object is fixed, the distribution of the transformed preferred rotation axis in the orientation space (θ₁, θ₂, θ₃) must be curl-free: Equation 38Any hypothetical neurons with non-zero curl can be ruled out by this condition (see below). For a gradient field, the zero curl is simply attributable to the equality of mixed second partial derivatives of the potential function, which holds also in higher dimensions. The equivalent path integral formulation is valid also in all dimensions: Equation 39along any closed curve in the six-dimensional space, where dl = (dx, dy, dz), and dθ = (dθ₁, dθ₂, dθ₃). Another equivalent formulation is that the potential function Φ can be constructed by the path integral: Equation 40which depends only on the end points, not on the exact path. Here ξ = (x, y, z, θ₁, θ₂, θ₃) is an arbitrary point in the parameter space, and ξ₀ is the value at a given initial point. Therefore, in the gradient theory, how the preferred translational direction and the preferred rotation axis of a neuron change with the view of a given object cannot be arbitrary but is highly constrained. This can provide testable predictions (see below).

Linear nongradient theory

A more general theory can be obtained by directly assuming a linear relationship between the firing rate and the components of the translational velocity v = (x˙, y˙, z˙) and the time derivatives of the angular variables θ = (θ˙₁,θ˙₂,θ˙₃). This yields the same tuning rule: Equation 41 where p = (p_x, p_y, p_z) and q* = (q^*₁, q^*₂, q^*₃) are arbitrary vector fields, not necessarily gradient fields, and Equations 26 and 32 are used in the last step. This tuning rule gives the same response properties predicted by the gradient theory for a single view of the object. The difference shows up when the view changes. The nongradient theory imposes no constraint on how preferred translational direction and preferred rotation axis should vary with the view of the object. The gradient theory is more restrictive, and therefore makes stronger predictions.

Change of reference center

Because the description of the same physical motion of a rigid object is ambiguous up to a parallel shift of the rotation axis (Fig.4), we have assumed in the above that the rotation axis always passes through the same reference point c = (x, y, z) in the object to ensure uniqueness of description. When a different reference center c′ is chosen, the form of the basic tuning rule in Equation 30 remains valid, but the preferred rotation axis is affected in a predictable way: Equation 42where v′ and ω′ are the translational velocity and angular velocity for the new reference centerc′, and: Equation 43 Equation 44are the new preferred translational direction and rotation axis. One can readily verify that Equation 42 is valid under Equations 43 and44, using the relations: Equation 45 Equation 46Therefore, changing the reference center of an object has no effect on the preferred translational direction of a neuron (Eq. 43), whereas the preferred rotation axis is altered systematically in a completely predictable manner (Eq. 44). These relations arise purely from the ambiguity of the description of rigid motion, and thus apply to both the gradient and the nongradient theories.

Summary

Simple assumptions have led to a general tuning rule for how the mean firing rate of a neuron should depend on the instantaneous motion of an arbitrary rigid object. For each given view of the object, the firing rate is predicted to be the sum of two terms, one for the translational motion component and one for the rotational motion component, both with cosine directional tuning and linear speed or angular speed modulation. In general, the preferred translational direction and the preferred rotation axis may depend on the identity of the object as well as its view. This tuning rule is a linear approximation to the geometry of rigid motion and therefore should obtain regardless of the exact computational mechanisms involved. In other words, this rule is expected to be a robust property for motion-sensitive neurons responding to realistic moving objects. When the view of the object changes, both the preferred translational direction and the preferred rotation axis of a neuron may change as well. The gradient theory provides additional constraints on such changes, whereas the nongradient theory imposes no further constraints. As a consequence, these two theories can be distinguished by further experiments. Finally, although the description of rigid motion is ambiguous up to a parallel shift of the rotation axis, the effects on the tuning rule are completely predictable and therefore convey no additional information about the response properties of a neuron.

Velocity component detectors

As illustrated in Figure6A, suppose the firing rate of an idealized neuron detects local motion on the image plane according to: Equation 47where f₀ is the baseline firing rate,p is the preferred direction of visual motion, andu is the local velocity on the image plane inside a small receptive field of the detector. This local velocity detector resembles some neurons in the middle temporal area (MT) of monkey, as discussed in the section after Equation 72.

• Download figure
• Open in new tab
• Download powerpoint

Fig. 6.

Examples of preferred rotation axis for velocity component detectors. Visual motion of a finely textured rotating object, here a sphere, is projected orthogonally onto the image plane. A, For a single local velocity detector with preferred direction p, the largest response is elicited by rotation around the preferred rotation axis q, which is on the horizontal plane and perpendicular to the vector r −c, with c representing the center of the sphere. B, When the response is the sum of two local velocity detectors with preferred directions p₁ andp₂, the preferred rotation axisq is perpendicular to the image plane.

This idealized neuron obeys the basic tuning rule in Equation 30, namely: Equation 48in response to a textured rigid object moving at translational velocity v and angular velocity ω. Here the preferred direction p is the same constant vector as in Equation 47, and the preferred rotation axis is given explicitly by: Equation 49where c is a fixed reference center in the object (center of the sphere in Fig. 6B), and r is the coordinate of the point in the object that happens to fall into the vanishingly small receptive field of the detector. As shown in Figure6A, the geometry of the situation is quite simple, with the two orthogonal vectors q and r −c both lying on the horizontal plane. For fixed angular speed, using q as the rotation axis maximizes the response of the velocity detector.

To derive these formulas, note that a point in the object with coordinate r has the velocity: Equation 50on the image plane (Fig. 6A). The desired equations can be obtained by inserting this into Equation 47 and using the vector identity: Equation 51Next, consider a higher-order neuron whose response is the sum of the outputs of several local velocity component detectors: Equation 52where p_i is the preferred direction of detector i, and u_i is the image velocity in its receptive field. This neuron also obeys the same basic tuning rule in Equation 48, with the preferred translational direction and preferred rotation axis given by: Equation 53 Equation 54For example, when two detectors are arranged as shown in Figure6B, the preferred rotation axis q is perpendicular to the image plane, whereas the preferred translational axis vanishes (p = 0) because the image size is constant under orthographic projection.

Finally, the basic tuning rule in Equation 48 still holds for image motion of a rigid object under a perspective projection, which projects each point (x, y, z) in the real world toward the observer at the origin (0, 0, 0), leaving an image at (X, Y) in the image plane at z = η: Equation 55(Longuet-Higgins and Prazdny, 1980; Heeger and Jepson, 1990). On the image plane, suppose the velocity component detector i is located at (X_i, Y_i) with preferred direction p_i = (p_i1, p_i2), then the preferred direction and preferred rotation axis become: Equation 56 Equation 57where ( _i, _i, _i) = (x_i − x, y_i − y, z_i − z)/z_i with (x, y, z) the reference center of the object, and z_i is the z -coordinate in real world for the physical point that happens to activate detector i. Unlike the orthographic projection in Figure 6, this mechanism allows a neuron to respond to looming or shrinking images, because it does not confine the preferred translational direction p to the image plane.

Spatiotemporal receptive field

Now consider motion-sensitive linear spatiotemporal receptive fields that obey the basic tuning rule in Equation 30 with a known potential function. Let I(X, Y, t) describe the intensity of an image at location (X, Y) on the image plane at time t, ignoring color and stereo. Suppose the firing rate of a neuron with linear receptive field F(X, Y) is linearly related to how fast the overlap between the image and receptive field is changing: Equation 58where the inner product is defined by: Equation 59This inner product can serve as the potential function postulated in Equation 29: Equation 60Therefore, the basic tuning rule in Equation 30 must hold true, and the preferred translational direction and rotation axis are: Equation 61 Equation 62following Equations 31-33. Here the partial derivatives are with respect to the implicit variables (x, y, z, θ₁, θ₂, θ₃) for the position and orientation of a moving object that generates the image.

More generally, consider a neuron with an arbitrary linear spatiotemporal receptive field G so that its firing rate is: Equation 63This becomes identical to Equation 58 for the kernel function: Equation 64where δ is Dirac delta function, which can be approximated by any narrow and normalized smooth function peaked at the origin. If Equation 64 is a reasonable approximation to the spatiotemporal receptive field, then the basic tuning rule in Equation 30 as well asp and q given by Equations 61 and 62 becomes valid. For nonlinear mechanisms, such as squaring (Adelson and Bergen, 1985) and normalization (Heeger, 1993), the above consideration may apply only after local linearization.

Existence of a global potential function

In all the concrete examples considered above, the basic tuning rule in Equation 30 holds true for each given view of an object. However, for a single view, the gradient and nongradient theories are indistinguishable. By assumption, the nongradient theory allows arbitrary preferred translational direction p and preferred rotation axis q. For a given view, the gradient theory can also generate any desired constant vectors p andq from the gradients of the following potential function: Equation 65where q* = qM is taken as a constant vector,c = (x, y, z) is the reference center of the object, and θ = (θ₁, θ₂, θ₃) describes the orientation of the object.

The gradient theory is globally correct only when a potential function exists for all views of the object. This is the case for the linear spatiotemporal model, where the potential function can be given explicitly (Equation 60). By contrast, for the idealized velocity component detector, a global potential function in general does not exist, as shown in Example 1 below.

Because the existence of a potential function does not depend on the choice of the coordinate system (see Appendix ), we only need to show that a potential function does not exist in the Euler angle space: (θ₁, θ₂, θ₃) = (θ, φ, ψ), assuming that the center of the object is fixed. In this three-dimensional space, a potential function exists if and only if the distribution of the transformed preferred rotation axis q* is curl free. Now consider two special examples that do not admit a global potential function:

Example 1: Constant preferred rotation axis fixed to the world. An explicit example is the model in Figure 6B, where the preferred rotation rotation axis q is the same regardless of the orientation of the spherical object. Here it is assumed that the velocity component detector has two vanishingly small receptive fields that can nevertheless detect the true local velocity components regardless of the orientation of the object. Without loss of generality, take the preferred rotation axis as a unit vector in the negative Y -axis: Equation 66and then compute its counterpart vector in the Euler angle space: Equation 67It is verified that curl q* as defined in Equation 38does not vanish. This proves that the desired potential function cannot exist for this hypothetical neuron (Fig.7).

• Download figure
• Open in new tab
• Download powerpoint

Fig. 7.

The gradient theory prohibits a neuron from having a constant preferred rotation axis in the physical space regardless of the view of the object, because this would allow the transformed preferred rotation axis (q^*₁, q^*₂, q^*₃) to have non-zero curl in the Euler angle space (θ, φ, ψ). Here the constant preferred rotation axis q = (0, −1, 0) is taken as the negative Y -axis. Only two-dimensional visualization is needed here because q^*₂ ≡ 0, and the vector field is independent of ψ.

Example 2: Constant preferred rotation axis fixed to the object. A possible example is a vestibular neuron receiving input from only a single semicircular canal without other influences such as that from the otolith. Then the firing rate has a cosine tuning with respect to a preferred rotation axis fixed on the head of the animal, regardless of the orientation of the head in the world (Baker et al., 1984; Graf et al., 1993). Without loss of generality, let the preferred rotation axis be a unit vector in the positive Z′ axis of the object (head), then in world coordinates this axis is: Equation 68so that its counterpart vector in the Euler angle space is: Equation 69Here curl q* is not zero, proving the nonexistence of the desired potential function.

Therefore, there are simple computational mechanisms that can violate the gradient theory. As shown above, the gradient theory prohibits a neuron from having a truly invariant preferred rotation axis fixed either to the world or to the object. These neurons, however, are allowed by the nongradient theory. In particular, the prediction of the gradient theory can differ from that of an optic-flow sensor. As another example, the preferred translational direction field for a small moving dot may also have non-zero curl when measured at different regions inside a large MST-like receptive field that has circular arrangement of local preferred directions (Saito et al., 1986). These idealized examples demonstrate that the global property of the gradient theory is quite restrictive, which, however, makes its prediction strong and refutable. Experiments could be performed to test whether the gradient theory accounts for the neuronal responses to three-dimensional object motion.

Spiral motion

No direct experimental data are available on how a neuron responds systematically to a realistic moving three-dimensional object with arbitrary translation and rotation. One closely related example is the broad tuning of some neurons to spiral visual motion, which may be generated plausibly by a large moving planar object facing the observer.

As shown in Figure 8, neurons in monkey visual medial superior temporal area (MST), which receive a major input from area MT, typically respond well to wide-field random-dot spiral motion patterns (Graziano et al., 1994). Most neurons in the ventral intraparietal area (VIP) are also sensitive to visual motion (Colby et al., 1993), and some have tuning properties to spiral motion similar to those in area MST (Schaafsma and Duysens, 1996), probably due to input directly from MST and/or integration of inputs from area MT. Area 7a is at a higher level than MST and might have more complex response properties for optic flows (Siegel and Read, 1997). In theory, it is possible to build an MST-like neuron from MT-like local motion inputs, even with position-invariance properties (Saito et al., 1986; Poggio et al., 1991; Sereno and Sereno, 1991; Zhang et al., 1993). Tuning to spiral motion was predicted based on Hebbian learning of optic flow patterns (Zhang et al., 1993) and by other unsupervised learning algorithms (Wang, 1995; Zemel and Sejnowski, 1998).

• Download figure
• Open in new tab
• Download powerpoint

Fig. 8.

Visual cortical neurons sensitive to spiral optic flow may provide evidence for the existence of preferred translation direction and preferred rotation axis for three-dimensional object motion. Here the average response of a neuron depended on the combination of radial and circular motion components, which varied systematically by changing local velocity directions in the moving random dot stimulus. The solid curves are circular normal fit, and the dashed curves are cosine fit. Data of the MST neuron are from Figures 8C in Graziano et al. (1994), and the VIP neuron data are from Figure 10 in Schaafsma and Duysens (1996), with permission.

To explain spiral tuning in terms of rigid motion, regard the environment itself as a large rigid object, moving relative to the observer. For the experiments mentioned above, the environment may be considered as a finely textured screen, oriented vertically, facing the observer. Translating this screen toward or away from the observer induces expansion or contraction, whereas rotating the screen around a perpendicular axis induces circular motion. According to the basic tuning rule in Equation 36, a neuron should respond to arbitrary motion of this screen with the firing rate: Equation 73where v is the translational speed, ω is the angular speed, and: Equation 74are constants, where α is the angle between the preferred translational direction and the actual translational direction, and β is the angle between the preferred rotation axis and the actual rotation axis.

To see why this accounts for spiral tuning, write Equation 73 in the equivalent form: Equation 75where s = (v, rω) is the spiral composition vector describing the stimulus, w = (P, Q/r) is the preferred spiral composition vectordescribing response properties of the neuron, r is a constant length, introduced to make v and rω of the same units, and γ is the angle between s andw. In the polar diagrams in Figure 8, the horizontal and vertical axes correspond to the two components of the stimulus spiral composition s, whose length: Equation 76was fixed during the experiments. It follows from Equation 75 that the response should fall off smoothly when the stimulus spiral composition becomes different from the preferred composition, in proportion to the cosine of the angle γ between them. This broad tuning to spiral motion is generally consistent with the data (Graziano et al., 1994; Schaafsma and Duysens, 1996), although a circular normal function in Equation EA17, with one more free parameter, may provide better fits than a cosine function (Fig. 8).

The above interpretation implies that firing rate should scale linearly with translational speed and angular speed independently of the spiral tuning curve. The responses of most MST neurons do indeed depend on speed (Tanaka and Saito, 1989; Orban et al., 1995), and many are monotonically increasing (Duffy and Wurtz, 1997a). It would be interesting to test quantitatively how well the linearity holds when averaged over a population of cells, especially for an ecologically relevant range of motion.

The tuning rule in Equation 73 also implies that the response should depend on the focus of expansion or the translational component in the optic flow, which also occurs for many MST cells (Duffy and Wurtz, 1995, 1997b). Adding a translational velocity vector to the stimulus corresponds to translating the stimulus screen sideways, which affects the angle α in Equation 74 and thus the response in Equation 73. Changing the translational direction and the rotation axis of the stimulus screen can alter both angles α and β in Equation 74 and thus the predicted response in Equation 73.

Motion-sensitive neurons in area MST may be used for purposes such as estimating heading or self-motion (Perrone and Stone, 1994; Lappe et al., 1996) or segmenting multiple moving objects (Zemel and Sejnowski, 1998). MST responses can be affected by various factors, including, for example, surround motion (Tanaka et al., 1986; Eifuku and Wurtz, 1998), disparity (Roy et al., 1992), eye position and movement (Newsome et al., 1988; Bradley et al., 1996; Squatrito and Maioli, 1997), vestibular input (Thier and Erickson, 1992), form cues (Geesaman and Andersen, 1996), the presence of multiple objects (Recanzone et al., 1997), and attention (Treue and Maunsell, 1996). Most experiments used simplified stimuli, although more realistic stimuli were tested recently (Sakata et al., 1994; Pekel et al., 1996). Because most of these examples contain parameters other than the object’s position and orientation, additional variables are needed to account for all of these effects in a model.

Further experimental test

Given all the contributing factors mentioned above, it is natural to ask how a neuron would respond to a more natural-moving three-dimensional object. A simple geometric stimulus is easier to specify and present but may lack important sensory cues needed to predict the response of a neuron to a natural stimulus. Our analysis relies on varying the translational direction and rotation axis and might provide a convenient basic description for response properties in terms of a preferred translational direction and a preferred rotation axis.

To test directly the basic tuning rule in Equation 30 or 36, one should present realistic images of a moving three-dimensional object to motion-sensitive neurons. The simplest way to test the theory is to oscillate slightly an object around a fixed axis. The oscillation should be sufficiently small so that salient visual cues are not occluded. For sinusoidal oscillations with frequency Ω and amplitude ρ: Equation 77and the angular speed is ω = dϕ/dt in Equation 36. Thus the theory predicts that the firing rate as a function of time should be: Equation 78where β is the angle between the actual rotation axis and the preferred rotation axis, q is a constant coefficient, and τ is the latency for visual response. The translational term in Equation 36 vanishes because there is no translational motion (v = 0). Systematically changing the orientation of the rotation axis (angle β) while keeping the view fixed allows the tuning function and the preferred rotation axis to be measured for the neuron.

Similarly, the response to translation in three-dimensional space could be tested by oscillating the whole object along a straight line: Equation 79The translational speed is v = dx/dt in Equation36, and the firing rate is: Equation 80where α is the angle between the actual translational direction and the preferred translational direction, and p is a constant coefficient. The preferred translational direction can be measured by systematically changing the translational direction (angle α) while keeping the view fixed.

For more efficient tests, the object could be rotated continuously with varying angular speed, covering all relevant views, first with respect to a fixed axis, and then systematically changing the axis. If the basic tuning rule is correct and the system is essentially linear, the tuning function and the preferred rotation axis could be computed for each view of the object. An even more efficient test is possible with a continuously time-varying rotation axis that generates tumbling movements of the object (Stone, 1998).

Eye position is one implicit factor that may affect the preferred translational direction and preferred rotation axis. The present theory allows an eye position effect but provides no additional constraints.

The linear response properties of a neuron for a given object are specified completely by its preferred translational direction, preferred rotation axis, and baseline firing rate for each given view of the object, as well as how these parameters depend on the view. All of these properties are experimentally testable and can be compared with the theoretical predictions in the preceding sections. For example, with the center of the object fixed, the curl-free condition for a given neuron should be tested by measuring its preferred rotation axis for four or more different orientations of the object. For a full test in six-dimensional space, both the preferred translational direction and the preferred rotation axis of the neuron should be measured for seven or more different positions and orientations of the object. See Appendix for further discussion.

If motion-sensitive neurons with similar tuning properties are clustered in the brain, then it might be possible to use functional magnetic resonance imaging techniques to test the predicted properties of the tuning rule in animal and human subjects using realistic images of moving 3-D objects as visual stimuli.