Abstract
Although the visual system has been extensively investigated, an integrated account of the spatiotemporal dynamics of long-range signal propagation along the human visual pathways is not completely known or validated. In this work, we used dynamic causal modeling approach to provide insights into the underlying neural circuit dynamics of pattern reversal visual-evoked potentials extracted from concurrent EEG-fMRI data. A recurrent forward–backward connectivity model, consisting of multiple interacting brain regions identified by EEG source localization aided by fMRI spatial priors, best accounted for the data dynamics. Sources were first identified in the thalamic area, primary visual cortex, as well as higher cortical areas along the ventral and dorsal visual processing streams. Consistent with hierarchical early visual processing, the model disclosed and quantified the neural temporal dynamics across the identified activity sources. This signal propagation is dominated by a feedforward process, but we also found weaker effective feedback connectivity. Using effective connectivity analysis, the optimal dynamic causal modeling revealed enhanced connectivity along the dorsal pathway but slightly suppressed connectivity along the ventral pathway. A bias was also found in favor of the right hemisphere consistent with functional attentional asymmetry. This study validates, for the first time, the long-range signal propagation timing in the human visual pathways. A similar modeling approach can potentially be used to understand other cognitive processes and dysfunctions in signal propagation in neurological and neuropsychiatric disorders.
SIGNIFICANCE STATEMENT An integrated account of long-range visual signal propagation in the human brain is currently incomplete. Using computational neural modeling on our acquired concurrent EEG-fMRI data under a visual evoked task, we found not only a substantial forward propagation toward “higher-order” brain regions but also a weaker backward propagation. Asymmetry in our model's long-range connectivity accounted for the various observed activity biases. Importantly, the model disclosed the timing of signal propagation across these connectivity pathways and validates, for the first time, long-range signal propagation in the human visual system. A similar modeling approach could be used to identify neural pathways for other cognitive processes and their dysfunctions in brain disorders.
Introduction
To understand the fundamental principles of the human visual system, it is crucial to know how visual signals are propagated, organized, processed, and transformed. Behavioral, anatomical, physiological, and theoretical studies have revealed that the visual system comprises of a complex hierarchy containing a mixture of “bottom-up” sensory-driven and “top-down” feedback mechanisms (Kastner and Ungerleider, 2000). The “bottom-up” theory suggests that visual sensory information is first translated into neural impulses by specialized photoreceptors located in the retina; and subsequently, the information is projected via the lateral geniculate nucleus of the thalamus to the primary visual cortex (striate cortex or V1), which then proceeds to higher-order cortical areas via the dorsal (occipitoparietal) and ventral (occipitotemporal) pathways, believed to underlie visual spatial and visual perceptual processing, respectively (Wang et al., 2012).
Evidence from macaque studies has revealed the presence of reciprocal feedback connections in the majority of the cortical pathways (Felleman and Van Essen, 1991). However, top-down (e.g., attentional) modulations are not confined to cortical processing (Wang et al., 2012) but can also occur within the thalamocortical networks, as demonstrated by single-cell studies on macaque (McAlonan et al., 2008), as well as fMRI studies in humans (O'Connor et al., 2002; Saalmann and Kastner, 2011). Moreover, invasive studies in animals have investigated the timing (latencies) of signal propagation in the visual cortex (V1 and V2) and throughout the visual pathways [including lateral geniculate nucleus, V1-V4, middle temporal (V5/MT), and medial superior temporal areas] (Nowak and Bullier, 1997; Schmolesky et al., 1998). These high-resolution studies have quantified the classical temporal hierarchy of information flow along the visual pathways. However, the single-unit recordings may not be sufficient to encode the information flow (neuronal populations are required) and may not readily be applied and investigated in human studies.
Noninvasive methods of evoked potentials are valuable for detecting the temporal profile of visual processing due to its reliability and high temporal resolution. In this context, several neuroimaging studies have investigated the spatiotemporal characteristics of neural generators in the visual system under a checkerboard pattern reversal process (Barnikol et al., 2006; Di Russo et al., 2007, 2012). For instance, using a combined visual-evoked potentials (VEP)-fMRI source analysis, Di Russo et al. (2007) showed that early pattern reversal VEP components (N75 and P100) are generated in the medial occipital cortex (V1) while N150 component is generated by contribution from several areas, particularly the parietal lobe [motion selective (MT/V5) area] as the major contributor and the mid-occipital (V3A) and ventral occipital (V4/V8) cortices as minor contributors. Further, Barnikol et al. (2006) investigated the anatomical identification of the underlying sources and the sequence of their stimulus-locked VEP activations. They used combined magnetoencephalography (MEG), magnetic field tomography, and probabilistic cytoarchitectonic maps of the visual cortex to remedy intersubject and interhemispheric variability. Interestingly, they not only found a consistent pattern of sequential and distinct activations (e.g., N75 was generated in V1/V2) but also found evidence of crosstalk among VC regions (between V1/V2 and V5 regions). However, these studies have provided only piecemeal evidence of the spatiotemporal dynamics of human visual processing, and an integrated functional anatomy remained unclear.
In this work, we address the above limitations by modeling the spatiotemporal dynamics of signal propagation across large-scale brain networks. In particular, we analyze the hierarchical cortical processing of the visual pathway using simultaneous EEG-fMRI data and the hypothesis-driven framework of dynamic causal modeling (DCM) (David et al., 2006). DCM models neural activity dynamics at the level of neuronal populations and thereby allows making an inference about connections within or between specified cortical source units. We introduce an optimal DCM model to analyze the signal timing of pattern reversal VEPs. Using a recurrent forward–backward model structure with seven identified source areas distributed in the brain, we provide strong evidence for these cortical source activities' contributions to the observed VEP. In addition, we show that, in higher cortical areas, the dorsal pathway has stronger effective coupling gain to primary visual cortex than that of the ventral pathway.
Materials and Methods
Participants
In this study, 12 male volunteers participated. All were in good health, had no history of neurological impairment, and had a normal or corrected-to-normal vision. The participants were 19–52 years of age (mean age 30 ± 9 years). A specific handedness was not required for this study. The subjects were recruited from both the university and hospital staff community. They provided their written consent after being described the nature of the study. Nobody was paid for his or her participation. The study was undertaken in accordance with the declaration of Helsinki, with full approval for the project sought from and obtained from the ethical review committee in St. James's Hospital, Dublin.
Data acquisition
fMRI acquisition.
The fMRI acquisition was performed continuously on a 3T MRI scanner (Philips Achieva 3.0T, Philips Medical Systems), located in the Centre for Advanced Medical Imaging, St. James's Hospital, Dublin, Republic of Ireland. The scanner was equipped with an 8-channel SENSE head coil, which was used for all experiments. The participants were comfortably positioned in the MRI scanner with their head inside the head coil. To reduce head movement, several soft pads were placed on both sides of the head. The subjects were instructed to maintain visual fixation on the screen and keep the number of eye-blinks to a minimum during the experiments.
For each subject, a high resolution T1-weighted structural scan was initially acquired for anatomical overlay on the fMRI images. Following this, 165 T2*-weighted, echo-planar, functional scans with a repetition time of 2 s were acquired, resulting in an overall acquisition time of 5.5 min for each fMRI experiment. The acquisition parameters are given in Table 1.
EEG acquisition.
The EEG data were acquired simultaneously with the fMRI scanning using an MR-compatible EEG amplifier (BrainAmp MR, Brain Products) and a 32-channel EEG cap (BrainCap MR, Brain Products). The electrodes were applied according to the international 10–20 system and were referenced to FCz with a common forehead ground. All electrode impedances were kept <10 kΩ. Because of the large gradient artifacts induced by the MRI scanner, the data were sampled at 5 kHz, with a resolution of 0.5 μV and a dynamic range of ±16.4 mV ensuring no amplifier saturation or aliasing effects. Furthermore, an additional electrode was placed on the back of each subject to record the ballistocardiogram artifact during the scanning. The digitized data were transmitted via an optical fiber out of the scanner room, avoiding any induced currents by the alternating magnetic field.
Experimental paradigm
The visual stimuli were generated by using the software Presentation 14 (Neurobehavioral Systems). The designed visual stimuli were back-projected by a color LCD projector (NEC MT1065, NEC) onto an Acryl screen behind the scanner bore while continuing fMRI acquisition. The subjects passively received the visual stimuli via a mirror integrated into the head coil. In addition, the stimuli were also displayed on a computer screen, which was located in the control room. In theory, the Presentation software provides stimulus timing at an accuracy of 1 ms. However, the temporal resolution was limited by the refresh rate of the computer monitor, which was set by default to 60 Hz. According to the Nyquist theorem, the stimulating power was uniformly distributed over a range from 0 to 30 Hz. Furthermore, Presentation controlled the synchronization between the MR scanner, the EEG system, and the computer screen. Therefore, different event markers were sent to the EEG recording system simultaneously with the onset of every acquired brain volume and every presented visual stimulus.
A black-and-white checkerboard pattern was reversed at a rate of 2 Hz in a block design consisting of 15 s stimulation and 15 s rest conditions. The checkerboard was composed of 8 × 8 checks. Each check subtended a visual angle of 0.6° both horizontally and vertically, while the whole checkerboard covered visual angles of 4.8°. During the rest periods, a gray screen was shown. The mean luminance of the checkerboard was kept constant throughout the entire experiment to avoid interfering flash VEPs. However, the absolute stimulus luminance and spectral composition could not be measured.
Data analysis
EEG data analysis.
The EEG signals were first corrected for gradient artifacts (due to the switching of the imaging sequence of the MR scanner) using the average artifact subtraction method (Allen et al., 2000) and for ballistocardiogram artifacts (due to the interaction between the static field of the MR scanner and the heart beats) using the PCA method (Leclercq et al., 2009). Next, the data were filtered in the range between 0.5 and 30 Hz and down-sampled to 256 Hz. Experimental trials were epoched from −50 to 250 ms relative to stimulus onset and baseline corrected using the average over the peristimulus time window. This period covered VEP components and was presumed to reflect almost all changes due to stimulus variations during a VEP process. The trials in which the signal on any channels exceeded ±80 μV were marked as eye blink or movement artifacts and rejected. Hence, an average 8.2% of trials were rejected from responses of each subject. Compared with analogous works (Warbrick and Bagshaw, 2008; Lalor and Foxe, 2009; Odom et al., 2010), we noticed a constant stimulus onset delay of ∼50 ms in our VEP components. Hence, we simply shifted the peristimulus time by 50 ms. Finally, to extract VEPs, we averaged trials using a robust averaging method (Wager et al., 2005).
fMRI data analysis.
The functional images were rereferenced to the anterior commissure area and spatially normalized into MNI space. Afterward, the images were smoothed with 10 mm FWHM Gaussian kernels using a third degree B-spline interpolation and voxel sizes of 2 × 2 × 2 mm3. A GLM analysis with a block design was exploited to evaluate the fMRI data at the subject level. A regressor was defined for stimulation response type and convolved with the canonical hemodynamic response function to estimate the BOLD activity. The movement-related parameters from realignment were also taken into account in the design matrix during GLM analysis. To identify the ROIs, the statistical parametric maps of group effects (assessed in the usual way using the summary statistic approach to random effects or intersubject analysis) were extracted from stimulation contrast map using a cluster-level threshold of p < 0.05, FWE corrected across the whole brain.
Asymmetric EEG-fMRI source localization
For each subject, we performed an EEG source localization constrained by spatial priors from fMRI data, so-called asymmetric integration (Henson et al., 2010). We used the fMRI contrasts obtained under stimulation condition to produce the 3-D volumetric image of the clusters as spatial priors. The clusters were projected in MNI space to the closest vertices of the canonical cortical mesh of each subject obtained from the T1-weighted structural scan. Given the generated fMRI contrasts, we inverted the VEPs using multiple sparse priors imaging source reconstruction with the greedy search scheme. In the imaging approach, the sensor data are projected into 3-D brain space using a large number of fixed dipoles distributed over the cortical sheet. This allows for exploration of larger regions of activity (Litvak et al., 2011). We selected greedy search algorithm for fitting and optimizing the parameters of inversion (Friston et al., 2008). This technique provides more plausible results in comparison with other hyperprior models, such as independent and identically distributed (minimum norm) and smoothness prior (LORETA) (Henson et al., 2009). We followed the following steps for performing EEG-fMRI source localization: (1) source space modeling using a cortical mesh consisting of 8196 vertices, (2) data coregistration, (3) forward modeling using boundary element method, and (4) inverting the VEPs using multiple sparse priors imaging method, optimized by greedy search algorithm and constrained by regions of interest from the fMRI results.
DCM
We used DCM for evoked potential responses to quantify the coupling changes (effective connectivity) and disclose the information propagation along the visual pathways. DCM relies on neural mass (i.e., neuronal population-based) Jansen-Rit formulation to explain multiple coupled source units (David et al., 2006). It can be specified by an input-state-output process where a stimulus-dependent phasic input drives the model to generate the source (pyramidal) activity, and then the system output is generated using a linear electromagnetic forward model (Daunizeau et al., 2011). As in a typical DCM framework, the cortical source model contains three subpopulations: excitatory deep pyramidal cells, inhibitory interneurons (e.g., basket cells), and excitatory stellate cells. Within a cortical column, both pyramidal cells and inhibitory interneurons are associated with the supragranular (II-III) and infragranular (V-VI) layers, whereas excitatory stellate cells are with the granular (IV). Their population dynamics are described by a collection of second-order differential equations that embody the parameters related to intrinsic (within area), extrinsic (between areas: forward, backward, and lateral) connections, synaptic, input, and conduction delays (David et al., 2006). The observation function is a lead field matrix (i.e., forward model) that is parameterized by the location and orientation of each source to account for the passive propagation of the electromagnetic field from the source (average membrane potential of pyramidal cells) to the sensor activity (EEG) with some observation errors (Kiebel et al., 2006). To complete the process of model inversion, DCM uses an iterative optimization procedure called variational Laplace to minimize the complexity (divergence) term and maximize the accuracy (energy) term (Friston et al., 2007). This inversion scheme returns a (free energy approximation to) log model evidence that is used for Bayesian model comparison (see below), where model evidence is accuracy minus complexity. Indeed, the variational Laplace is a generalization of expectation maximization algorithm where in each step the ensemble of a set (i.e., density) instead of a point is estimated. The parameters of the mean-field under a fixed-form Laplace (i.e., Gaussian) assumption are factorized into a set of Gaussian densities to facilitate the maximization process.
We selected to model the VEP data during the poststimulus period 0–250 ms under the checkerboard condition. For computational expediency, we elected 8 singular value decomposition components corresponding to spatial modes to explain the data in a reduced form. This data reduction also prevents the modeling of artifacts (Kiebel et al., 2006). We inverted models for each subject under the Bayesian optimization scheme and evaluated them using Bayesian model selection that uses free energy as a measure of model evidence. To make group inference, we compared them by their exceedance probabilities (EP) under the random effect assumption that takes the subject variability into account (Stephan et al., 2010). After the model selection, we inspected the temporal profile of pyramidal source activities that are thought to be the major contributor to the local field potentials or EEG signals.
Software note
We used FASST.2 (montefiore.ulg.ac.be/∼phillips/FASST.html) to remove the BCG and gradient artifacts from EEG data, EEGLAB version 10.0.0.0b (sccn.ucsd.edu/eeglab), and custom MATLAB scripts to visualize the results in 3-D space, and SPM12 version 6225 (www.fil.ion.ucl.ac.uk/spm) to perform the DCM and source analyses. All toolboxes are implemented in MATLAB (The MathWorks).
Results
EEG and fMRI data analysis
The group-averaged VEPs to the checkerboard stimulation over scalp electrode locations exhibited large-magnitude responses localized to the visual cortices (occipital lobe) with a peak at 125 ms after stimulus (Fig. 1A). In particular, the VEPs during the stimulation condition characterized by the classic triphasic pattern consisted of an initial negative component at ∼100 ms (N1), followed by a pronounced positive peak ∼125 ms (P1) and a second negativity with a peak latency of ∼170 ms (N2) after stimulus, consistent with previous works (Warbrick and Bagshaw, 2008; Lalor and Foxe, 2009; Odom et al., 2010).
Figure 1B shows the results of EEG source localization constrained by spatial priors of fMRI, which gave rise to seven prominent source areas. The areas consist of the thalamic area (TA), left/right visual cortex (LVC/RVC), left/right inferior temporal cortex (LIT/RIT), and left/right posterior parietal (LPP/RPP). The area detected by source localization in the deep brain is the thalamic area based on its proximity and its temporal profile (Fig. 1B; see Propagation of neural activities along the visual pathways). In the localization procedure, the sources were identified as they peaked with the largest conditional estimates of evoked power [sum of squared source activity over the selected time, i.e., maximum intensity projection (MIP)]. The sources in the order of MIP values were the TA, RVC, LVC, RPP, LPP, RIT, and LIT areas. The whole-brain GLM analysis of fMRI data similarly revealed clusters in the following: RVC, t = 13.97, 1303 voxels; LVC, t = 13.23, 871 voxels; RPP, t = 4.45, 124 voxels; LPP, t = 3.73, 102 voxels; RIT, t = 3.4, 15 voxels; LIT, t = 3.1, 10 voxels; and TA, t = 2.5, 8 voxels. Some minor activations were found in the frontal areas of 6 of 12 subjects (t = 2, 5 voxels) that might be related to attentional modulation (Corbetta and Shulman, 2002). Both integrated EEG-fMRI and GLM analysis of fMRI data suggested stronger activity and spatially extensive responses in the right hemisphere with respect to the left.
Model selection
Figure 2 shows the anatomical structure of the three plausible neural circuit models investigated, each containing seven source areas based on the above reconstruction results. For simplicity, we did not consider the bilateral frontal areas in our model hypothesis due to having lower active voxels (GLM analysis) and low MIP values (source localization). Moreover, to reduce the model parameters, we did not explicitly develop a neural model for the thalamic activity but directly fed its dipole source activity to the cortical neural model as inputs. In other words, in contrast to standard implementations of DCM, we replaced the simple (Gaussian) thalamic input with empirically reconstructed thalamic responses that contributed to the measured signal and drove higher (cortical) sources [the SPM code (“spm_erp_u.m”) was edited to accommodate empirical inputs]. The thalamic source activity was estimated in the approximate coordinates of x = 0, y = −5, z = 10 in MNI space. The structural models investigated differ in terms of extrinsic forward and backward connections. Model 1 is a purely feedforward (“bottom-up”) cortical hierarchical model, in which the visual stimuli are processed in an ascending fashion along the pathways (Fig. 2A). In this hypothesis, the visual information that is bilaterally entered from the retina into the thalamus relays to the contralateral side of the primary visual cortex and then continues along the two parallel ventral (inferior temporal) and dorsal (posterior parietal) pathways. Model 2 is similar to model 1 but has an extra feedback connection from PPs to VCs on the ipsilateral side (Fig. 2B). Model 3 represents a recurrent forward–backward process with two extra backward connections from PPs and ITs to VCs on the ipsilateral side (Fig. 2C). DCMs were inverted for each subject under the checkerboard condition. The group Bayesian model selection gave rise to a substantially higher exceedance probability for model 3 ((EP3 = 0.83) > (EP1 = 0.16) > (EP2 = 0.01)). Hence, this model was selected for our subsequent analyses.
Propagation of neural activities along the visual pathways
We inverted the most probable DCM model for the grand mean (12 subjects) VEPs under stimulation condition. Figure 3 shows the reconstructed results at both source and sensor levels. At the source level, responses in the thalamic area presented a temporal profile with a main positive peak ∼70 ms after stimulus (Fig. 3A). Driven by the thalamus, the VC areas then produced a profile, with similar temporal characteristics at the peaking region as the cortical sensor level VEPs (i.e., both peaked at 125 ms after stimulus in Fig. 3A,C). The IT and PP sources revealed amplitude-reduced and temporally delayed type of responses of the VC source activity (Fig. 3B). Interestingly, we found that the PP sources have greater activities than the corresponding IT sources in both brain hemispheres for VEPs from grand average (Fig. 3B) and for most of the individuals (9 of 12) (Fig. 4). Moreover, the right sources were stronger than the corresponding left sources. This could perhaps be caused by the slightly higher biased attention in the left hemifield (for further evidence, see Effective connectivity analysis). In sensor space, the first principal component of estimated responses after 50 ms after stimulus indicated a close match with observed measurements for stimulation condition (Fig. 3C). The projected 3-D scalp topographic maps further verified the accuracy and consistency of the estimates obtained from the model (Fig. 3D).
After analyzing the grand mean VEPs, we investigated the source signal propagation timing of the individual's VEP. Consistent with group responses in Figure 3, subjects generally provided similar evidence for the dominance of dorsal over ventral pathways and right over the left hemisphere. Figure 4 shows the estimated VEP source activities of 4 of 12 subjects during stimulation condition. As evident in Subjects 1, 7, and 12, the activity of right sources in the visual cortex as well as in two streams is greater than the contralateral side (left). This effect can be also seen in the results of the Subject 3, but with the direction reversal. In total, we found a dominance of the right hemisphere in the results of 9 of 12 subjects.
Effective connectivity analysis
In addition to analyzing the source signal propagation, we investigated the effective connectivity among the considered brain regions. Specifically, based on the inverted optimal model for grand mean VEPs, we evaluated the effective couplings among cortical regions and the posterior correlations. Figure 5 summarizes the results in terms of the percentage change in coupling gains (values in bold with yellow background), the occurrence probability for each modulated connection (values in parentheses with yellow background), and interdependency among extrinsic connections that are overlaid on the structural model hypothesis (values with orange background). The conditional probability >0.95 is approximately equivalent to a significant p value of 0.05 (Stephan et al., 2008). It should also be noted that, when discussing about a modulation or gain of a connection, the modulation is in relation to the prior expectation. In DCM, effective connectivity parameters are modeled as log-scale parameters, which we report in terms of a proportional increase or decrease, relative to the prior mean.
In terms of forward connections or feedforward process, the strongest coupling was that from thalamus to right and left visual cortices with gains of G = 3.61 and 2.21, respectively. This was followed by two coupling gains, of G = 1.91 with occurrence probability of 1.00 from RVC to RPP, and G = 1.60 (0.82) from LVC to LPP, where both were stronger than the coupling gains of left and right VC to RIT (G = 0.93 (0.94)) and LIT (G = 0.82 (0.91)) on the ipsilateral sides. Next, we analyzed the conditional correlation (normalized auto-covariances) matrix among the forward connections. A positive conditional correlation suggests that the mirror pathways between the two hemispheres are symmetrical, whereas a negative conditional correlation indicates asymmetry. We found positive values in connections between RVC-RPP and LVC-LPP (r = 0.45). Hence, this may infer the equal propagation of signals between the hemispheres for the VC-to-PP pathway. In contrast, a negative correlation was found for the VC-to-IT pathway (r = −0.30), suggesting asymmetrical signal flow along the ventral streams between the hemispheres.
In terms of “top-down” or backward connectivity effects, enhanced effective connectivities were found from PPs to VCs in both right (G = 1.61 (1.00)) and left hemispheres (G = 1.42 (0.95)). These backward connectivities were stronger than those from IT to VCs (to right VC, G = 0.71 (0.63); to left VC, G = 0.73 (0.68)) and more probable on both sides. This implies a greater contribution of dorsal than ventral pathways perhaps due to attentional top-down process (Corbetta and Shulman, 2002). The backward connectivity is generally weaker than the forward connectivity, consistent with previous works (Felleman and Van Essen, 1991; Bastos et al., 2015).
We further examined the differences in extrinsic connectivity parameters of the most likely model at the group level under Bayesian parameter averaging (Stephan et al., 2010). Table 2 shows the average connectivity estimate of forward and backward connections along with their SDs and posterior or conditional correlations. The Bayesian parameter averaging generally gave rise to consistent results with the effective connectivity analysis of grand mean VEPs in Figure 5. For instance, the coupling gains of the connections on the right hemisphere were stronger than the corresponding coupling on the left hemisphere. Moreover, both forward and backward (or feedback) connections from the VC to PP areas were stronger than that of VC to IT areas. However, it is important to note that one should not overinterpret the conditional or posterior correlations in terms of underlying neurophysiology. This is because they reflect conditional dependencies that can be introduced by inefficient estimation. In other words, if two model parameters change to produce very similar differences in sensor space, they generally have negative posterior correlations.
Discussion
We have validated, in an integrated way, the relative timing of long-range signal propagation of neural sources along the human visual pathways. This was achieved by applying a DCM approach to simultaneous EEG-fMRI data under a pattern reversal VEP task. The computational model was constrained by known neurobiology and the EEG-fMRI data. The specific brain regions were modeled based on the inverse EEG source localization aided by fMRI spatial priors.
Our source estimation and connectivity model approaches to VEPs characterized the temporal profile of the neural activities of visual processing in the human cortex. In particular, we made use of the temporal profile of the reconstructed thalamic activity as inputs to the cortical neural models. This provides a more realistic inputs than typically used in DCM (David et al., 2006). In addition, the activities of VCs at the source level showed consistency in terms of the temporal characteristics with the first principal component of estimated scalp measurements (both peaked at 125 ms; Fig. 3A,C), confirming the VC sources as the major contributors to the scalp measurements. Importantly, the model quantified the timing of propagation of signals from thalamus to the primary VCs, followed by the PPs (dorsal pathway) and the ITs (ventral pathway). This sequence of activated signals was also found to be attenuated, which is expected in signal propagation within neural networks (Vogels et al., 2005). Therefore, our current study provides an integrated large-scale picture of visual signal propagation within the human brain.
Using effective connectivity analysis, we have shown that the simple visual passive stimuli, such as pattern reversal (checkerboard), activates the dorsal pathway while suppressing the ventral pathway. This ventral pathway inhibition may also arise from higher level cognition (e.g., from the frontal cortex), although our EEG-fMRI analysis only picked up weak frontal activation (Fig. 1D). This confirms that the frontal cortex is less involved in passive stimulus-driven (e.g., pattern reversal) than active task-driven experiments (e.g., controlling the location of attention or memory activity) (Corbetta and Shulman, 2002). The model also showed that backward interactions or “top-down” influences are higher in the dorsal than ventral pathways, perhaps hinting the presence of “top-down” processing in the former than the latter. This is consistent with previous experimental studies on humans and monkeys, where the parietal cortex was shown to be generally involved in attentional “top-down” processing whereas temporal cortex contributes more to object recognition processing (Kastner and Ungerleider, 2000; Corbetta and Shulman, 2002).
Based on our GLM-fMRI and DCM-EEG analyses (Fig. 5), we found a bias favoring the right hemisphere, which can be originated from nondominant (left) hemifield. This functional attentional asymmetry has been linked to “pseudoneglect” (Bowers and Heilman, 1980). This behavior (i.e., visual right hemisphere cerebral lateralization or leftward spatial bias) has also been observed in recent neuroimaging studies (Hougaard et al., 2015; Loughnane et al., 2015). However, we suggest that such behavior may potentially be more systematically investigated using model-driven connectivity analysis techniques used in this study.
The results and approaches in this work have several important implications. The computational model, which was tied closely to experimental (neuroimaging and neurobiological) data, has helped to validate and quantify the specific timing and propagation of brain activity, and provides an important fundamental step toward an improved understanding of human visual processing. Based on the effective connectivity analysis, the model not only indicates bottom-up sensory processing but also suggests top-down processing, despite the simplicity and passivity of the task.
There are previous studies using the DCM approach to investigate the hierarchical processing of sensory stimuli in the human cortex, wherein the interactions among forward and backward (reciprocal) connections are used to explain the underlying mechanism of a cognitive phenomenon. Earlier works by Büchel and Friston (1997) and Friston et al. (2003) attempted to explain the attentional modulation of evoked responses in early visual cortices based on connectivity changes in fMRI sources. Later, Garrido et al. (2008) used the DCM for evoked potentials to show that the mismatch negativity responses during an auditory oddball task can be explained by a message-passing hypothesis called predictive coding. Under this hypothesis, neural responses to unpredictable stimuli are enhanced (relative to responses due to predictable stimuli) due to larger prediction error caused by feedback loops in higher level sensory hierarchies. In another study, Chen et al. (2009) applied DCM for induced (spectral) responses measured with MEG during face perception. Based on evidence for functional asymmetries between forward and backward connections, they concluded that the forward connections in the lower cortical area led to higher (γ) oscillations, whereas their interactions with backward connections in higher cortical areas led to lower (β) oscillations.
However, an important aspect missing in the above and related studies is the signal timing of source activities, which along with connectivity change, can provide a better understanding of the spatiotemporal dynamics of signal propagation across large-scale brain networks. Such timing information could potentially be important to delineate cognitive mechanisms and identify brain disorders. Also, our experimental task is sufficiently simple and passive without involving complex higher cognitive processing, which may mask the bottom-up visual processing. Hence, this can provide a “baseline template” for investigating more complex visual tasks. Another advantage of our work is making use of multimodalities (simultaneous EEG and fMRI) to improve source localization, which leads to better model accuracy. Furthermore, the input signals to our cortical neural models are based directly from the source (i.e., thalamic) activity and not model-based, thus, more realistic input temporal profile while reducing the number of model parameters.
In general, the DCM and effective connectivity analysis approaches can generally be extended to other data types (e.g., MEG), other stimuli, and cognitive processing, especially when timing and signal propagation are crucial (David et al., 2006). A relevant area to be applied would be visual perceptual decision-making tasks (Philiastides and Sajda, 2007; Ostwald et al., 2012), where cortical neural mass or mean-field decision-making models have already been developed (e.g., Wong and Wang, 2006; Niyogi and Wong-Lin, 2013). As mentioned above, dysfunctions in signal propagation have been implicated in neurological and neuropsychiatric disorders (Näätänen et al., 2011). For example, multimodal sensory integration has been suggested to be important in falls in elderly people (Lewkowicz and Ghazanfar, 2009). Our approach, which can elucidate the temporal profiles and propagation of cortical and subcortical source activities (Fig. 3), together with the coupling gains among the brain regions (Fig. 5), may potentially be useful for detecting signal processing disruptions caused by brain disorders (or lesions), such as visuospatial neglect (Kastner and Ungerleider, 2001; Corbetta and Shulman, 2011) or schizophrenia (Lalor et al., 2012).
Footnotes
The data acquisition was funded by Centre for Advanced Medical Imaging Centre from the Health Research Board Ireland institutional grant. V.Y. was supported by the Ulster University Vice-Chancellor's Research Scholarship Award. G.P. and K.W.-L. were supported in part by Northern Ireland Functional Brain Mapping Project 1303/101154803, funded by InvestNI and the University of Ulster. We thank Muhammad Naeem for helping with the initial phase of the data analysis; Donnell J. Creel for useful discussions and comments on an earlier version of the manuscript; and two anonymous reviewers for their highly constructive feedback to help improve the manuscript.
The authors declare no competing financial interests.
- Correspondence should be addressed to either Dr. Vahab Youssofzadeh or Dr. KongFatt Wong-Lin, School of Computing and Intelligent Systems, Ulster University, Northland Road, Derry–Londonderry, BT48 7JL, UK. youssofzadeh-v{at}email.ulster.ac.uk or k.wong-lin{at}ulster.ac.uk