Abstract
Can the adult brain assimilate a novel, topographically organized, sensory modality into its perceptual repertoire? To test this, we implemented a microstimulation-based neuroprosthesis that rats used to discriminate among infrared (IR) light sources. This system continuously relayed information from four IR sensors that were distributed to provide a panoramic view of IR sources, into primary somatosensory cortex (S1). Rats learned to discriminate the location of IR sources in <4 d. Animals in which IR information was delivered in spatial register with whisker topography learned the task more quickly. Further, in animals that had learned to use the prosthesis, altering the topographic mapping from IR sensor to stimulating electrode had immediate deleterious effects on discrimination performance. Multielectrode recordings revealed that S1 neurons had multimodal (tactile/IR) receptive fields, with clear preferences for those stimuli most likely to be delivered during the task. Neuronal populations predicted, with high accuracy, which stimulation pattern was present in small (75 ms) time windows. Surprisingly, when identical microstimulation patterns were delivered during an unrelated task, cortical activity in S1 was strongly suppressed. Overall, these results show that the adult mammalian neocortex can readily absorb completely new information sources into its representational repertoire, and use this information in the production of adaptive behaviors.
SIGNIFICANCE STATEMENT Understanding the potential for plasticity in the adult brain is a key goal for basic neuroscience and modern rehabilitative medicine. Our study examines one dimension of this challenge: how malleable is sensory processing in adult mammals? We implemented a panoramic infrared (IR) sensory prosthetic system in rats; it consisted of four IR sensors equally spaced around the circumference of the head of the rat. Each sensor was coupled to a microstimulating electrode placed in the somatosensory cortex of the rat. Within days, rats learned to use the prosthesis to track down items associated with IR light in their environment. This is quite promising clinically, as the largest demand for sensory prosthetic devices is in adults whose brains are already fully developed.
Introduction
In recent decades, sensory prosthetic systems such as cochlear implants (Wilson et al., 1991) have helped tens of thousands of patients regain functional levels of sensory processing (Macherey and Carlyon, 2014). In the future, these peripheral prosthetic devices will likely be complemented by systems with higher throughput that directly activate neurons in the CNS, in particular the cerebral cortex (Dobelle and Mladejovsky, 1974; Dobelle, 2000; Coulombe et al., 2007; Srivastava et al., 2009; Davis et al., 2012; O'Doherty et al., 2012; Andersen et al., 2014; Lewis et al., 2015). Such central neuroprosthetic systems could eventually become as routine in the treatment of sensory deficits as deep brain stimulation has become in the treatment of Parkinson's disease (Deuschl et al., 2006; Fuentes et al., 2009; Santana et al., 2014; Yadav et al., 2014) and even some psychiatric disorders such as depression (Mayberg et al., 2005).
To work properly, cortical sensory neuroprosthetics will require that the adult brain be plastic enough to continuously process real-time streams of synthetic information sources, and then use the extracted information to guide appropriate behavioral responses. Classically, the highest levels of cortical plasticity have been observed during early development (Frost and Metin, 1985). For instance, in a set of rewiring experiments in newborn ferrets, primary auditory cortex was induced to process visual information when visual information was routed to A1 (Sur et al., 1988). Following this procedure, individual neurons in rewired A1 exhibited visual response properties typically associated with neurons in V1, such as center-surround receptive fields (RFs; Frost and Metin, 1985; Sur et al., 1988; Horng and Sur, 2006), and rewired A1-mediated visual behaviors in these transformed animals (von Melchner et al., 2000).
Recently, we reported similar findings in adult rats in which information from a single infrared (IR) sensor was delivered, via cortical microstimulation, to primary somatosensory cortex (S1; Thomson et al., 2013). After a month of training with this hardware, rats readily discriminated the location of IR sources in the environment in exchange for a water reward. This cortical prosthesis completely bypassed the native sensory transducers of the animals, delivering sensory information never experienced by these animals into a region of cortex already devoted to processing somatosensory information. This type of sensory enrichment system was recently extended when the output of a geomagnetic compass was coupled to stimulating electrodes in S1 of rats placed in a T-maze (Norimoto and Ikegaya, 2015).
Overall, these experiments show that the adult mammalian brain can adaptively absorb completely new sources of sensory information delivered directly to cortical tissue. They also demonstrate the viability of using rodent models in the development of cortical sensory prostheses. However, several questions left open by these initial studies will have to be addressed for the development of more realistic neuroprosthetic systems. For instance, how does the brain respond to these qualitatively new information sources? Will animal performance improve with the addition of more sensors, or would this additional information be a source of confusion? Does the spatial distribution of the stimulating electrodes matter? If different combinations of sensors were inactivated, would behavioral performance degrade gracefully, the way natural sensory systems do?
Many of these questions would be impossible to ask using a neuroprosthesis containing only a single transducer. Thus, in the present study we have extended the IR discrimination paradigm to include four IR sensors distributed to provide a full panoramic (360°) representation of the surrounding IR environment (Fig. 1A). The rats were not confused by the additional sensors and stimulating electrodes, but learned the task an order of magnitude faster than when equipped with just a single IR sensor. In animals trained to discriminate IR sources, individual S1 neurons exhibited both IR and robust tactile evoked responses. These and many other neurophysiological adaptations described below underlie the emergence of a multimodal IR tactile representation in the adult rat S1, allowing animals to perform an IR discrimination task at nearly 100% accuracy.
Materials and Methods
Training and behavioral task.
All experiments were conducted on female Long–Evans rats ∼10 weeks old (200–224 g; Harlan Sprague Dawley). For behavioral training, rats were trained in a circular chamber (50.8 cm diameter) with four reward ports that were situated 90° apart. The reward was given through a poke hole that was fit with a photo beam. Each port was additionally fit with a visible broad-spectrum LED, and an infrared LED (Opto Semiconductors; 940 nm peak emittance; range of nonzero emissions, 825–1000 nm). The IR sources had an angular width at half-maximum of 120°). A custom-made pushbutton was affixed to the floor in the center of the chamber, which rats pressed to initiate each trial (27 mm diameter; Fig. 1A).
Initially, 17 rats were pretrained in a simple visual discrimination task, in which they were placed in a circular chamber with four water ports that contained, in a vertical arrangement, a visible-light LED, an infrared LED, and a water spout (Fig. 1A). In the initial pretraining, rats learned a very simple visual discrimination task in which one of the four visible lights turned on, and animals were rewarded when they selected that port. Selection was indicated by breaking the photobeam in front of the water spot.
Once the performance of the animals crossed 85% correct on the visible light (see Materials and Methods) version of the task (which took 9 ± 3 d), we then prepared them for the IR discrimination training. Once animals learned to use the center button to start a trial, they normally performed well above 95% correct. First, four groups of eight microelectrodes were chronically implanted in the whisker representation—barrel cortex—of both S1s of each rat (see Stimulating electrode implantation surgery). Each microelectrode group was associated with a different IR detector (Fig. 1B). We chose this approach as the most natural way to associate each IR detector with a topographic homolog in the rat S1. So, for instance, the front-right IR detector was intended to stimulate the region of S1 corresponding to the right-front whiskers, corresponding to the left anterior barrel field in S1.
Next, we retrained rats on the IR version of the original discrimination task, replacing visible light with IR light. Thus, for rats to be rewarded in a trial they now had to track the IR beam all the way to the port from which the IR beam was being emitted. For each session, the four IR detectors were snapped into place on the head of each animal via a magnetic seal. The output of each IR detector was independently connected to the activity in the stimulation channel in the appropriate quadrant of S1, so, for instance, if the head of the animal was oriented so that the right-front IR detector faced the active IR source, then the left-front region of S1 would be highly activated (Figs. 1C–E). The frequency of stimulation in each stimulating channel varied approximately exponentially with IR levels in its corresponding IR detector (Thomson et al., 2013). We excluded two animals from subsequent analysis: one developed an infection at the site of implantation, and the acrylic head cap came off the other after surgery.
Initial current levels for microstimulation were found by stimulating in the lightly anesthetized animal a week after surgery. For each of the four locations, we determined the minimal threshold for at least one pair of stimulating electrodes, defined as the smallest current we could deliver (20 pulses at 200 Hz) and evoke a clear response in S1 (as determined by listening to responses in an audio monitor).
We then trained animals in the same behavioral chamber, but replaced the visible light with intracortical microstimulation (ICMS) proportional to the level of IR light detected by a sensor. A trial was counted correct when the rat poked the port with the activated IR light. After the animals were able to perform the task solely with IR light and met the criterion of reaching 85% correct, we sometimes conducted a series of additional experiments, which are described below (e.g., remapping experiments).
All behavioral and microstimulator controls were performed using custom MATLAB scripts using the data acquisition toolbox (run with NIDAQ PCI-7742 card, National Instruments).
Stimulating electrode implantation surgery.
We implanted the electrodes bilaterally in S1 under full anesthesia. To initiate the anesthesia, rats were first put to sleep with isoflurane (Isothesia, Henry Schein Animal Health). For final anesthesia, we used a ketamine (Ketaset, Fort Dodge)-xylazine (AnaSed, Akorn Animal Health) anesthetic with 100 mg/kg ketamine and 0.06 mg/kg xylazine. During surgery, supplemental doses (33% of the initial dose, administered intramuscularly) of ketamine were provided when necessary. Throughout surgery, we delivered at least 3 ml of saline (Hospira) every 2–3 h, injected subcutaneously. Further, 0.02 mg/kg atropine (Atropine, West-Ward Pharmaceuticals) was given within the first hour and then every 2–3 h until the animal regained consciousness. Surgeries took between 3.5 and 6 h, excluding recovery from anesthesia.
Coordinates for the craniotomy were taken with the stereotactic tools relative to bregma at 2.5 mm posterior and 5.5 mm bilateral. Electrodes were lowered 1.5 mm into the brain at an angle of 10°. Electrode pairs in one penetration were 300 μm apart, so the electrode tips of each pair were positioned at 1.5 and 1.2 mm below the surface of the cortex (Thomson et al., 2013). The craniotomy was covered with eye ointment before the electrodes were fixed in place, and the craniotomy was sealed using dental cement (Perm Reline, Coltene/Hobbylinc).
After surgery, the rats were given at least 7 d to recover. During this time, we provided them with Tylenol for pain relief and monitored their weight. The Duke University Institutional Animal Care and Use Committee approved all surgical and behavioral methods.
Four IR sensor neuroprosthetic.
The electrodes, implanted in the somatosensory cortex, formed the first part of the neuroprosthetic. Electrodes were implanted in each hemisphere (Fig. 2B). In both hemispheres, the layout was identical and consisted of 16 electrodes and one ground electrode. Preliminary studies revealed that the thresholds for evoking behavioral responses were lower when two electrodes were paired in the same penetration of the cortex (Thomson et al., 2013). For our stimulating and recording electrodes, we therefore joined two 42 μm stainless steel polyimide insulated microwires.
Panoramic IR discrimination, related to Figure 2. A rat locates the source of the IR light with the neuroprosthetic. The movie is captured with an IR-sensitive camera. Each trial is initiated when the rat presses the center button and a choice is made by poking the nose into the correct port. Audio signal is a direct feed from the microstimulator, and is not available to the rat.
The second part of the neuroprostheses consisted of an attachable cap that included four IR sensors. The phototransistor in the IR sensors (Lite-On) had a peak spectral sensitivity at a wavelength of 940 μm with a range of sensitivity of 860–1020 μm. The sensor had a 20° width at half of its maximum sensitivity. The four infrared sensors were placed 90° apart from each other on a horizontal plane (Fig. 1B). The head cap could be plugged into the connector (Omnetics Connector Corporation) on the head of the rat and was sealed in place with a small magnet.
To more fully characterize the response profile of the system, the IR sensor response as a function of IR sensor angle and position in the chamber was measured off-line in the behavioral chamber. This was done by affixing the panoramic IR sensor array onto a stepper motor (Portescap) that was rotated in 15° increments, with the response of the IR sensors measured at each angle to yield a full IR response profile at a given position in the behavioral chamber (Fig. 1D). Such measurements were made at each position in the behavioral chamber, at 1 inch spatial resolution. The main differences in responses at each position were the angular preference, and width at half maximum of the response. The latter is shown as a function of position in the chamber (Fig. 1E).
The in-house-made stimulator used a bipolar stimulation with charge-balanced, biphasic pulse trains (Hanson et al., 2012). The frequency of stimulation depended on the intensity of the detected IR light. We used seven different frequencies, ranging from 0 to 425 Hz (Thomson et al., 2013).
Neural recording.
The basic setup for recording neuronal activity has been described in detail previously (Wiest et al., 2008; Thomson et al., 2013). Briefly, in the channels we did not use for stimulation, we recorded neural activity using the Multichannel Acquisition Processor (MAP; Plexon). To stimulate and record simultaneously, the wires connected to the stimulator bypassed the head stage, while the other channels went directly to the head stage. Sorting of neural data is also described previously (Wiest et al., 2008). Briefly, in addition to template-based on-line sorting, all voltage traces around a threshold-crossing event were saved for off-line sorting. For off-line sorting, we used clustering in principal component space and refractory periods to assign data either as single units or multiunits.
Sensor ablation.
To confirm that the animals used all four IR sensors and not only a subset of them, in six rats we ran whole sessions during which we deactivated the ICMS in one or multiple locations. The number of ICMS locations that were deactivated ranged from zero (all active) to four (all deactivated). We used different combinations of deactivated channels to test for preferences of a side or an orientation. The deactivation of all four ICMS sites was a necessary control to test whether the rats relied solely on the stimulation itself or whether they used other cues to guide their behavior. We used three to five different ablations in one session, with at least 40 trials per ablation; the order in which the ablations were presented was randomly determined. A session in this experiment consisted of 140–220 trials, depending on the number of different ablations used in the session, 20% of which were trials with visible light.
Varying angles between reward ports.
As a common measurement of a sense (modality), we were interested in the spatial resolution of the ability of the rats to discriminate IR sources. We moved the reward ports closer together so that the angle between them within a session was 90°, 60°, 45°, or 30°. A session consisted of 60 trials at two or three angles, with 20% of trials with visible light. The order in which the angles were presented was randomly determined. This was done with six rats.
IR sensor and S1 remapping.
To test the ability to adapt to a new input structure, and determine whether the rats were using the spatially distributed information provided by the prosthesis, we changed the connection between the four sensors and the four ICMS sites. We used three different permutations: front-back flip, where the front sensors controlled the stimulation in the posterior ICMS sites and vice versa; left-right flip, where the left sensors stimulated the left hemisphere and vice versa; and a diagonal flip, a combination of the left-right flip and the front-back flip. In the diagonal flip, the front right sensor controlled the ICMS in the posterior barrel cortex in the right hemisphere, and all other channels were connected respectively. One remapping was applied consistently until the rats learned to use the new mapping. The threshold of learning was a performance of >85% correct in IR-only trials. In these experiments, to start, 100% of the trials had only the IR light as a cue. If the rats did not reach a performance of 25% after three sessions, we included trials with visible light for a few sessions. This was necessary only when the very first remapping was a diagonal remapping.
Task-irrelevant microstimulation control animals.
To control for stimulation-induced plasticity, we trained and implanted two other animals that received stimulation that was not informative for any task. They were treated in the exact same way as the animals in the IR discrimination task, except they performed only the visible version of the task. At random times during this task, they were presented with stimulation patterns collected from rats that performed the actual task.
Histology.
At the end of the experiments, seven rats were killed to confirm the electrode positions. The rats were anesthetized with 1 ml of Euthasol and perfused with saline, followed by a 3% paraformaldehyde in 0.1 m phosphate buffer. The brain was removed from the skull, and the subcortical structures were carved out. The cortex was carefully flattened for 5 h in a 3% paraformaldehyde solution. After the cortex was flattened, it was fully fixed in a 4% paraformaldehyde solution overnight and then dehydrated in a 30% sucrose solution for another 24 h. Cryostat sections of 40 μm were stained with cytochrome oxidase to show the barrel fields, as described previously (Thomson et al., 2013).
Analysis.
Data acquisition and analysis was performed in MATLAB using custom-written code, the Statistics Toolbox, and the Circular Statistics Toolbox (Berens, 2009). Here we discuss specific techniques we used.
The stimulus population vector P in response to the microstimulation pattern S = <f1, f2, f3, f4> is as follows:
where vi is a vector pointing in the direction of IR detector i (e.g., the top right detector location is represented by <1, 1>), and fi is the frequency of stimulation in stimulating channel i.
We used a bootstrap analysis to generate the 95% confidence ellipses around the mean of two-dimensional (2D) quantities. Namely, we took 10,000 bootstrap samples from the set of all values of the quantity, and calculated the mean of each bootstrap sample. We then fit the distribution of such bootstraps with a 2D Gaussian with the covariance matrix of the bootstrap samples, oriented in the direction of the eigenvectors, with the major and minor axes that determine the size of the ellipse determined by the covariance matrix.
Multiple-comparison tests were performed in MATLAB to determine individual differences when a general ANOVA was significant. The test consisted of multiple t tests with Bonferroni correction for multiple comparisons.
In calculating the IR-RFs, we took the mean response for each stimulus delivered, used linear interpolation to fill in gaps for where there were no stimuli presented, and then smoothed the resulting map using a 2D Gaussian filter with a width = 30 and σ = 10.
To extract head direction and initial movement at trial onset, manual video analysis was used. Head bearing was determined by the nose direction with a graduator on the monitor screen. For the initial movement, only trials where the IR light came from the left or right side of the animal head were used (45–135° or 225–315° relative to head bearing of 0°). A trial was counted as having head movement if the animal swiveled its head >10° after IR source onset; trials without such angular head movements (16% of trials) were discarded.
Results
Behavioral task
We trained 15 animals to discriminate the location of an IR source (see Experimental procedures). Briefly, each animal was trained in a location-discrimination task in a cylindrical arena that contained four water ports affixed to its border; in addition to a water spout, each port contained a visible and an IR light (Fig. 1A; Thomson et al., 2013). In the task, a light in a single port turned on, and, to receive a water reward, the animal had to break a photobeam in that port. Initially each rat performed the task with visible light only. Once they reached 85% correct, they were implanted with four groups of stimulating electrodes in four quadrants of S1 (Fig. 1; Experimental procedures). In subsequent sessions, we used the output of four IR sensors, which were evenly distributed around the head of the rat to control the stimulation frequency of stimulating electrodes in four quadrants of S1 (Fig. 1B; Experimental procedures). When one of the IR sensors was closer to, or oriented toward, an IR source, it evoked higher-frequency microstimulation in its corresponding S1 stimulating channel (Fig. 1C–E).
Methods for rat IR discrimination training. A, Schematic of the behavioral chamber used for the task. Four ports are arranged symmetrically around the inner surface of a large (24 inch) cylinder. Each port contains a nose poke, an IR light, and a visible light. In the middle of the floor is a button the rat pushes to initiate a trial. B, Topographical organization of information from four IR detectors in rat S1. The IR sensors were placed 90° apart from each other, with each sensor coupled to a different stimulating electrode pair in S1. Note that the representation of the left whiskers is found in the right hemisphere, but the front-facing whiskers are still represented on the anterior part of the barrel field in S1 (N1 through N4 represent the four locations of neuronal stimulation). C, Stimulation frequency depended on IR intensity in each sensor. The intensity of each IR light was translated into different stimulation frequencies, in real time, in its corresponding stimulation channel. D, Polar plot showing the response of each IR sensor as a function of angle in the chamber, when the sensor array is at a fixed position in the chamber, relative to a single activated IR source. The red point on the edge indicates the relative location of the IR source. E, Full-width at half-maximum (FWHM) of the response profiles as a function of position in the chamber. The red point is the position of the active IR source, while the FWHM is the mean FWHM of all four sensors at the given position (see D). As you move further way, or oblique, to the source, the response profiles become narrower. The black point indicates the position of the data represented in D.
Performance in IR discrimination task
Thirteen animals quickly learned to discriminate IR sources using the synthetic panoramic IR signals delivered to S1. Figure 2A shows the learning curve of these animals, in which the percentage correct is plotted as a function of the number of days using the IR neuroprosthesis. On average, the animals surpassed the 85% correct mark within 3.75 ± 0.45 sessions (mean ± SEM). The mean best performance (the mean of their best five sessions) of the rats was 98.0 ± 0.51% correct. As rats learned the task, the latency, or time between ICMS onset and selection of the port, decreased significantly from 5 to <2 s (Fig. 2B). Figure 2, C and D, compares the learning rates of rats when trained with four IR sensors, with the data from a previous study in which animals were equipped only with a single IR sensor (Thomson et al., 2013). The rats wearing the new panoramic IR neuroprosthesis crossed the 85% correct threshold significantly faster than animals implanted with a single IR detector (3.75 ± 0.45 sessions with four IR sensors vs 22.3 ± 7.62 sessions for rats with one IR sensor).
Learning to use distributed IR information to discriminate IR sources. A, The mean percentage of correct responses for the first sessions from eight animals (mean ± SEM). The dotted line marks the 85% threshold. Also, see Movie 1. B, The latency of the response (time between stimulation onset and animal breaking a photobeam in one of the ports) over different sessions, from the same rats as in A. C, Comparison of rats with one IR sensor (gray) and rats with four IR sensors (black) implanted on the head. The black line is the same data shown in A. The one sensor group consists of three animals and is averaged with a sliding window over 3 d. D, The number of sessions to reach 85% correct was significantly different for one vs four IR detectors. Rats with four IR sensors needed 3.75 ± 0.45 sessions to reach 85% correct, while rats with only one IR sensor needed 22.3 ± 7.62 sessions. A two-sided t test shows a significant difference between both groups (p = 0.0019). E, F, Circular histogram of head direction at trial onset relative to the last visited port. E, The head direction before they learned to use the IR implant. F, The head direction after they learned to use the IR implant. Ports are located at 0°, 90°, 180°, and 270°. Before they learned, they faced 178° away of the last visited port, and after they learned they faced 155° away from that port. G, Performance vs task difficulty. The schematics above show the layout of the task chamber for the four-IR sensor implant. Rats with four IR detectors got >85% correct even when the ports were 30° apart. The figure compares the performance of rats with four IR sensors (N = 6; red line) vs one IR sensor (N = 3; green line). The graph shows the mean percentage of correct responses, and the error bars show the SEM. Rats with four IR sensors showed a trend of decrease in performance in IR-only trials for smaller angles (p = 0.132, ANOVA). The performance of rats with only one sensor dropped significantly for 45° and 30° (p = 0.0008 and p < 0.0001, respectively, multiple-comparison test). For 45° and 30°, rats with four IR sensors performed significantly better than rats with only one IR sensor (p < 0.0001 and p = 0.0029 respectively, multiple-comparison test). H, The performance decreased significantly as more sensors were deactivated (p < 0.0001, one-way ANOVA). The graph shows the mean/SEM of behavioral performance, the dotted line is chance performance (25%). Data are pooled from six animals, disregarding which channels were deactivated. The performance with only one of the four sensors was at 52.8 ± 2.07% (percentage correct ±SEM) and therefore was still above the chance level of 25%, regardless of which sensor was still active. When all sensors were deactivated, the performance dropped to 14.5 ± 3.77%. A post hoc test showed that, with the deactivation of each sensor, the performance decreased significantly. I, Behavioral performance when specific channel combinations were inactivated shows that certain channels show a stronger influence on the performance than others. Inactivating of single sensors showed a trend of reduced performance but was not significant. Surprisingly, when both front-facing sensors were deactivated, the drop in performance was not significant (p = 0.363, multiple-comparison test). In contrast, a significant decrease occurred when both back, both left, or both right sensors were deactivated (all p < 0.0001 when compared with all sensors active condition, multiple-comparison test). No difference was found between the left and right sensors (p > 0.99, multiple-comparison test). When three of four sensors have been deactivated, the performance dropped significantly (all p < 0.001). For comparison of p values across all ablation conditions, see Table 1.
Unlike rats implanted with a single IR detector, animals implanted with our panoramic IR neuroprosthesis did not sweep their heads laterally in an attempt to sample IR signals from the surrounding environment (Thomson et al., 2013). Instead, once they received an ICMS with IR light onset, these rats often turned directly toward the correct port, likely because the four IR sensors provided them with a more complete perspective on the IR sources (see Discussion).
To investigate the animals' behavior when using the new implant, we recorded the head direction when the animals initiated each trial. Specifically, we calculated the head bearing relative to the last poked port. Here we found that before the animals used the IR implant, they typically faced directly away from the previous port poked (178° away). Interestingly, after they learned to use the implant, they faced 155° away (Fig. 2E,F). This change with learning was significant (p < 0.0001, nonparametric ANOVA), and both distributions were significantly different from a uniform distribution (p < 0.0001 and p < 0.0001, respectively, Raleigh test).
In the next set of experiments, we varied the task difficulty by moving the IR sources closer together in the chamber, randomly setting the angles between ports to 90°, 60°, 45°, or 30° (Fig. 2G, inset, arena layout icons). Surprisingly, while we observed a slight decrease in performance as the ports were closer together, this was not statistically significant in rats wearing a four-IR detector neuroprosthesis (Fig. 2G; p = 0.132, ANOVA). This is in contrast to the results in our previous study with a single sensor (Fig. 2G; Thomson et al., 2013, their Figs. 1, 2).
Ablating IR sensors
How would the animals respond if we shut off different combinations of IR sensors, so that rats did not have the full panoramic view of the IR environment? We examined this in six animals, in which different combinations of IR sensors were inactivated on randomly selected trials (see Experimental procedures). We call such inactivations sensor ablations.
On average, with each additional sensor ablated, the performance of the rats dropped significantly (p < 0.0001, one-way ANOVA; Fig. 2H). The p values for the comparisons of performance in all ablation conditions are indicated in Table 1. With the inactivation of one channel, performance dropped significantly when compared with performance when all IR sensors were active (91.4 ± 0.80% correct with all sensors vs 86.1 ± 0.95% with only three sensors active; p = 0.0209, multiple-comparison test). With each additional channel deactivated, the performance decreased significantly (all p < 0.0001, multiple-comparison test). The performance reached 74.7 ± 1.7% with two sensors deactivated, and 52.8 ± 2.07% with three channels deactivated. The rats were still able to perform the task above chance level (25%), with only one of four IR sensors activated. When all four sensors were deactivated, so that the animal was receiving no information from the prosthetic, the performance dropped drastically, down to 14.5 ± 3.8%, which is below chance (Fig. 2H). Note that performance can be lower than chance when rats do not make a behavioral selection, and allow the trial to time out.
The p values for comparison of performance in ablation experiments
Figure 2I summarizes the performance for every particular sensor ablation experiment, showing that some IR sensor combinations were more important than others for task performance. Surprisingly, we observed that ablating individual back-facing IR sensors caused a larger decrement in performance than the small but insignificant decrease observed when a single front-facing sensor was inactivated (Fig. 2I). Similarly, ablating both rear-facing channels led to a decrease in performance to 70.0 ± 2.1% (p < 0.0001, multiple-comparison test), which was significantly larger than the decrement observed after ablating both front-facing channels (p = 0.002, multiple-comparison test; Table 1). With just a single sensor active, even if it was a front sensor, performance was still significantly above chance (p < 0.05; see Discussion).
Finally, when the two sensors on one side of the animal were ablated, the performance dropped significantly (to 70.1 ± 3.6% when the right side was ablated and to 65.5 ± 5.3% when the left side was ablated; p < 0.0001, multiple-comparison test). There was no significant difference between the left-only and right-only conditions (p > 0.999, multiple-comparison test).
Topography remapping
Our sensory neuroprosthetic system gives us complete control over the mapping between the IR sensor and the stimulating electrode. This allowed us to examine the rat's sensitivity to the spatial distribution of IR signals within S1. More specifically, once animals had learned the task with the normal configuration, how did they react when that mapping between the IR sensor and stimulating electrode was changed (Fig. 3A)?
Disrupting the topographic IR map temporarily impairs task performance. A, Once animals had learned the task to criterion (at least 85% correct), we performed a series of remapping experiments in which we changed the mapping from IR sensor to stimulating electrode. This panel illustrates the four different mapping conditions we used, such as switching hemispheres, front-back switches, or both. B, C, Examples from two rats run through multiple remappings. After the rats were proficient with one mapping of the stimulation, a new mapping was introduced. Each remapping type is depicted in a different color. The performance on the day before the first remapping is depicted by the green dot at session 1. Interestingly, all animals that started with the diagonal remapping stayed at chance performance, and in those animals we added visual cues as an additional aid (this is indicated by the black x). Also, see Movie 2. D, A summary slide of all learning curves for the first remapping and all subsequent remappings. Data are collected from nine rats in three groups. Each group started with a different type of remapping (left-right, front-back, or diagonal). Subsequent remappings were chosen pseudo-randomly for all rats in all groups. One-dimensional remappings (left-right and front-back) were learned significantly faster (p < 0.0001, multiple-comparison test) and without the guidance of visible light. No difference in learning speed was found between the left-right and front-back remappings (p = 0.74, multiple-comparison test). It took 3.83 ± 0.7 sessions to learn the first one-dimensional remapping. The first diagonal remapping took 11.7 ± 2.3 sessions. All rats needed visible light as guidance when confronted with a diagonal remapping initially. We started visible light when the rats performed worse than 10% on their fourth day of remapping. E, Analysis of error patterns in the first 20 trials to during left-right remapping. For those trials in which IR lights were activated to the left or right of the animal, the plot shows the percentage of trials in which the animal turned toward the correct light (see Materials and Methods). Performance was significantly worse during the first 20 trials on the first day of remapping compared with the day before the first remapping and the third day of remapping (p = 0.0001 and p = 0.0003, respectively, ANOVA). F, After learning the first remapping, all subsequent remappings were learned in 2.17 ± 0.2 sessions, regardless of the type of remapping (p = 0.79, ANOVA). G, Number of sessions the rats needed to reach the 85% threshold for different remapping conditions. After rats learned one remapping, they learned subsequent remappings significantly faster (p < 0.0001, multiple-comparison test). This suggests that a metaplasticity mechanism is in play. H, Mean ± SEM performance on the first five sessions for animals trained with normal mapping from the start (N = 8) and those trained with diagonal mapping (N = 5 animals). The animals trained with diagonal mapping performed significantly worse (p = 0.007, two-sided t test). I, Learning curve for normally mapped vs diagonally mapped animals, using the same groups of animals as in G. The dotted line marks the 85% threshold.
Left-right remapping example, related to Figure 3. The connection between the sensors and the stimulation channels was switched between left and right (Fig. 3A). During the first trials, the rat was expecting the normal mapping and turns in the wrong direction when an IR light is activated to the left or right. After training (in this case, ∼18 min), the rat has begun to adapt to the new mapping. (The lights on the left side of the arena are used for off-line image processing and are not relevant for the task.)
We examined the following three topographic remappings in nine animals: left-right remapping, in which we connected IR sensors to stimulating channels in the ipsilateral cortical hemispheres instead of the usual contralateral loci (Fig. 3A); front-back remappings, in which the front IR sensors project to the barrels that represent the rear whiskers and vice versa (Fig. 3A); and third, diagonal remappings combine the first two remappings, completely reversing the usual topography in both dimensions. For convenience, hereafter we will call the first two examples—left-right and front-back—one-dimensional (1D) remappings, whereas the diagonal remapping is classified as a 2D remapping.
The impact of all remappings was immediate and profound. For instance, in left-right remappings, when an IR source appeared on the right side of the animal, the rat would promptly move to the left and select the incorrect port. Despite such understandable miscalculations, rats quickly started to recalibrate their performance during their first sessions with 1D remappings.
Quantitatively, Figure 3B shows the behavioral performance for a series of remappings in one animal. It started with a front-back remapping (its performance the day before remapping is indicated by the green dot). The performance of the animal dropped during the first session, but it soon recovered as it learned the new relationship between IR sensors and stimulating electrodes. After animals successfully relearned the task following an initial remapping, we then tested their ability to adapt to subsequent remappings. In this case, after the front-back shuffle, we then remapped the left and right IR sources, resulting in a 2D diagonal remapping. Again, the animal learned the correct association within a few sessions (Fig. 3B). The performance of the animal on subsequent remappings is indicated in Figure 3B.
Curiously, none of the three animals that started with a diagonal remapping learned the task unaided: we used visual cues to help them properly recalibrate their maps. Figure 3C shows a representative animal that started on the diagonal remapping. The rat struggled on the first three sessions with below-chance performance. Indeed, all animals stayed at below chance level (25%) when first introduced to the 2D remapping, and ultimately lost motivation in the task. To keep them motivated, we supplemented ICMS with visible light to guide them to the correct port (visible trials: sessions in which we used visible trials are indicated in Fig. 3C). Over the next few sessions, we included a decreasing proportion of visible trials. Such visually guided recalibration of the mapping was never necessary for 1D remappings, but it was needed for all three animals to learn the 2D remapping.
Figure 3, D and E, shows the pooled data for each remapping group. No difference was found between the two 1D remappings (p = 0.74, multiple-comparison test): on average, it took them 3.83 ± 0.7 sessions to learn the first 1D remappings (Fig. 3D, blue and green lines). On the other hand, rats that started with the diagonal remapping took 11.7 ± 2.3 sessions to relearn the task, even though they had the help of the visible light (Fig. 3D, orange). This was a significant difference in learning rate (p < 0.0001, multiple-comparison test). These results show that the rats clearly used the spatial information available in the distributed IR prosthesis.
We also examined, in more quantitative terms, the pattern of errors after left-right remappings. Specifically, we measured whether the initial head movement was toward or away from the IR source in the first 20 trials of the first remapping session (Fig. 3F). This measure was also made for the session preceding the first remapping, and on the third day of the remapping. As expected, the animals turned in the opposite direction directly after remapping significantly more often than before (p = 0.0001, ANOVA). After learning the remapping, the animals turned directly toward the IR source, indicating that they had fully adapted (p = 0.0003, ANOVA).
Interestingly, once a rat experienced its first remapping, it learned subsequent remappings significantly faster, suggesting the existence of a central metaplasticity mechanism (see Discussion). Figure 3E shows the learning curve for all secondary remappings in animals after they reached criterion (85% correct) on their first remapping. Interestingly, the speed of learning for secondary remappings did not vary for the 1D versus 2D diagonal cases (p = 0.79, ANOVA). Rats reached the 85% correct threshold in 2.17 ± 0.2 sessions. Not only were secondary remappings learned faster than initial diagonal remapping (p < 0.0001, multiple-comparison test), but they were also learned faster than the first 1D remappings (Fig. 3F; p = 0.048, multiple-comparison test).
The previous results address the effect of remapping in an already trained animal, but what if we change how they are trained initially? That is, is it important that they start with the most “natural” mapping in which (for example) the front-right detector activated stimulates electrodes in the region of S1 representing the front-right whiskers (Fig. 1)? To answer this, we trained five rats with the diagonal mapping condition from day 1. The overall percentage correct in the first five sessions for rats initially trained on the normal mapping was 64 ± 7%, while in the diagonal-mapped animals it was 40.0 ± 10% correct, showing a significant difference (Fig. 3H; p = 0.0007, two-sided t test).
While the diagonally mapped animals took longer to learn the task (Fig. 3I), they did ultimately reach a best five-session performance of 98.4 ± 0.43% correct, which did not differ significantly from the best five-session performance of the 11 normally mapped rats (p = 0.718, two-sided t test). Interestingly, the average response latencies of the diagonally mapped rats (2.56 ± 0.168 s) never matched those of the rats with the normal topography (1.73 ± 0.07 s; p = 0.0011, two-sided t test).
Note that, while there was a weak but significant correlation in learning rates between the learning time of the visual task and the IR task (p = 0.0494, Spearman correlation), we found no further correlations between learning rates in subsequent subexperiments.
Stimulus population vectors
In the next series of experiments, we recorded neuronal activity in S1 while rats performed the IR discrimination task. As shown in Figure 4A, to work around the stimulus artifact that occurs during electrical microstimulation, we modified the task to stimulate S1 intermittently. Specifically, after sampling the IR level in a sensor, we stimulated for 75 ms at the corresponding frequency, followed by a 100 ms quiet period. During this period of quiescence, we were able to record S1 neuronal activity. This was done in all four stimulation locations, as depicted in Figure 4B. Quantitatively, every 175 ms there was a four-element vector of frequencies S = <f1, f2, f3, f4> delivered to four regions of S1 (Fig. 4B,C). Figure 4B represents five such stimuli, depicted at times T1 … T5.
Quantifying the stimulus projected into S1. A, Illustration of change in protocol used when simultaneously stimulating and recording. When just stimulating, we updated the stimulation frequency in each channel every 50 ms based on the IR levels in its corresponding detector (Fig. 1F). In animals in which we stimulated and recorded, we sampled the IR levels approximately every 175 ms, and delivered stimulation for 75 ms in each stimulating channel, and then turned off the stimulators for 100 ms to facilitate recording. B, Each stimulation pattern can be represented by a sequence of four-element vectors S of microstimulation frequencies in four locations in S1. The panel shows five simplified patterns of microstimulation. C, Conversion from 4D vector representation of stimulus to the 2D population vector, where each time point (T1–T5) corresponds to one of the five time points in B. The actual numbers (0–100) are arbitrary and are used only to clarify the procedure. D, The same four population vectors from C are depicted as locations in the 2D population vector space. Again, this is for illustration only, to facilitate understanding of simple stimulus population vectors, and does not include real data. All four sensor location vectors are indicated for reference, as are five population vectors (labeled T1 … T5). The black square is the outline of the bound of all possible stimulus population vectors, which are discussed more in D. E, The set of all possible population vectors (gray points), with the set of all actual population vectors over all recording sessions overlaid (red points; 19 sessions in three animals, with 171 ± 53 trials per session and 14 ± 0.53 stimuli per trial). Note that the actual vectors (red) tend to cluster in the medial anterior portion of the space, with relatively few population vectors on the extreme lateral and caudal regions of the space of population vectors. The range from 0 to 850 Hz results from the maximum stimulation frequency of 425 Hz per channel, which can sum up to 850 Hz in the two-dimensional transformation. F, The set of actual population vectors delivered in a single session, with those stimuli in the same trial connected by lines. Red indicates error trials, while green shows correct trials. Lighter circles represent earlier stimuli in the trial, and more saturated colors indicate later stimuli. The black spots represent the mean value of the first three and last three stimuli in the session. The early stimuli hovered around the origin, while the final three stimuli tended to be closer to the rostral pole, indicating the two front IR channels (and stimulating electrodes) were highly activated. In this session. There were 6.0 bits of entropy contained in the stimulus set. The light gray circles indicate the set of all possible population vectors as before, and are included for reference. G, Count histogram of the number of stimuli shown on each trial, over all sessions. The distribution is approximately bimodal, with the peak centered at ∼25 indicating those trials in which the animal let the trial time out without poking, and reached the upper limit of net stimulation. H, PSTH in response to the first three and last three stimuli in the same session, as indicated in D. The stimulation periods are indicated by the numbers 1–6, and time is relative to the onset of the first stimulus. Note that this PSTH does not represent the response to the exact same stimulus each time: while the first three stimuli tend to cluster around the origin and the last three tend to cluster in the mediorostral region, but there is a good deal of variability (D). This is simply a way to get a quick overview of the types of responses we see. We indicate the potential break in time between the first three and last three stimuli on the x-axis with the slashes.
To compactly depict the electrical stimulation delivered to the brain in one of these distributed bursts, we represented the full 4D representation of the stimulus into a more easily visualizable 2D space using a stimulus population vector. This population vector is defined exactly like those popularized in the motor control literature (Georgopoulos et al., 1986). The stimulus population vector is the weighted sum of the location of each IR sensor (Fig. 4D), where the location of each sensor is weighted by the stimulation frequency in its corresponding stimulating electrode. Quantitatively, the stimulus population vector P in response to S = <f1, f2, f3, f4> is as follows:
where si is a vector pointing in the direction of IR detector i (e.g., the rear-left detector location is represented by <−1, −1>.
For instance, when the front two IR detectors are strongly activated, the population vector is at the top of the graph (Fig. 4A–D, time point T5). Quantitatively, the rear-left IR sensor (sensor 3) is represented by the vector <−1, −1>, and if we stimulate at 200 Hz in its corresponding stimulating channel, then the contribution from sensor 3 to the population vector will be <−200, −200>. The contribution from all four sensors is summed to yield the full population vector. Figure 4, B and C, works through an artificial example of how to visualize five such vectors. This is meant for illustrative purposes to help explain the population vector concept. Figure 4E shows, in red, the set of all actual population vectors observed in all 20 sessions in three rats. The set of all possible population vectors in our stimulus set is shown as gray points in Figure 4E. The upper bound in each direction is indicated with the black diamond. We stimulated between 0 and 425 Hz in an individual channel, and these limits determined the upper bounds of 850 Hz for any individual population vector (Eq. 1).
Note the relative paucity of such vectors in the posteromedial region, which exists because the animal rarely backs up into an IR source. That is, the two rear sensors are rarely activated strongly together: the animal does not approach the target moving backward, but typically orients toward the target and then runs forward.
To show how the stimulus population vectors change over time within a trial, Figure 4F shows the set of population vectors that occurred in one session, with contiguous substimuli from a trial connected by lines (the red points are error trials, green are correct trials). The lighter points/lines represent the stimuli that occurred earlier in the trial, and darker rendering represents the later stimuli. Note the characteristic shape of the curves: stimuli tended to start around the origin (<0, 0>) in the space of population vectors, and as the animal approached the correct IR port, the population vector shifted to the top part of the graph, indicating that the front two IR sensors were strongly activated. The mean of the first and last three stimuli in the session are shown in black. On average, 14 ± 0.53 stimuli occurred on each trial (Fig. 4G).
Neuronal peristimulus time histogram during the execution of the IR detecting task
We used multiple measures to quantify the neuronal responses in S1 during a session. Figure 4H shows one measure: the mean neuronal population peristimulus time histogram (PSTH) in response to the first three and last three stimuli in a session. The times of actual stimulation are numbered 1–6 in the PSTH, where the first 3 are aligned to the onset of the cortical stimulation and the last 3 to the offset. During the 75 ms stimulation periods, the PSTH is zero because of stimulus artifact, and then there is a 100 ms period of neuronal activity recording after the stimulus ends (Fig. 4A).
The set of all such neuronal population PSTHs obtained in a given recording session is shown in Figure 5, where the location of each recording microelectrode is also indicated. We consistently found the IR-evoked neuronal responses to be broadly distributed across the recording microelectrodes in all four quadrants of S1. The inset on the right side of Figure 5 depicts the mean neuronal response as a function of the total number of stimulating channels activated. This graph shows that the larger numbers of spikes were produced by S1 neurons until two IR channels were activated. Beyond that point, when three or four IR channels were activated, S1 neuronal firing tended to decrease, though this change was not significant (ANOVA, p > 0.05).
S1 PSTH in response to stimulation sequences. The mean response to the first three, and last three, stimulus vectors, as depicted in Figure 4F. The arrows from the brain show the location of the recording electrodes. Each PSTH follows the conventions shown in Figure 4H. The inset shows the spike count z-score as a function of the number of stimulating channels activated. This is a typical profile, with maximum response occurring when two channels are coactivated. The z-score was calculated by comparing the spike count after stimulation, and we used the mean/SD of spike counts before trial onset (calculated over all trials; data are from N = 3 animals, 20 recording sessions, 299 multiunit recordings). Even when no stimulating channels are active, the response is typically higher than in the baseline period before the trial begins.
Infrared neuronal receptive fields in the rat S1 cortex
While the PSTH in response to the first and last three stimuli gives a convenient visual representation of neuronal activity during the task, it is also a relatively incomplete sample of the full range of neuronal firing patterns. To give a full representation of how S1 neurons responded to IR stimuli, we introduced a new measurement: the IR-RF.
The IR-RF is the mean S1 neuronal firing response magnitude (spike count), as a function of the stimulus population vector, for all of the IR stimuli shown in a session. Figure 6A shows all of the IR-RFs for the same S1 neuronal units depicted in Figure 5. The mean value for each stimulus is color coded in a contour plot defined over the set of stimuli, where red lines depict peak response values. For instance, unit 019i has a peak at the most anterior/medial portion of the stimulus space, indicating that it responds maximally when the two front IR sensors are strongly activated (Fig. 4), while unit 031i has its IR-RF center located in the posterior-right region of the space.
IR receptive fields. A, Mean number of spikes in response to different stimuli. The stimuli are represented as population vectors (Fig. 6), and the mean spike count in the 60 ms time window is represented by isocontour lines (low rates in blue, high in red: see color bar). Same session and units as used in Figure 5. The small red point on each contour plot shows the point where the receptive field is maximized. B, Distribution of locations of RF peaks in all units with the normal mapping (179, multiunits total). They tend to be located along the midline, and rostral. That is, they tend to favor the activation of the two front IR channels. The histograms on the sides show the marginal frequencies. The red spot shows the mean, and the circle around it shows the 95% CI for the mean (see Materials and Methods). The mean RF centers are also indicated for the four different recording locations. C, Distribution of locations of actual stimulus vectors presented in all experiments (see Fig. 4E). They tend to be located along the midline, and rostral, just like the receptive field centers. The histograms on the sides show the marginal frequencies. D, Bar plot showing the relative number of neurons with preference for stimulation from pairs of stimulating electrodes vs individual stimulating electrodes. E, Proportion with preference for pairs is significantly greater than chance (p = 0.003, χ2 test). Angular preference of all neurons are from B. Angular histogram shows the relative number of neurons with IR-RF centers at different angular locations. F, Peak response magnitude (z-score, with mean/SD spike count calculated from a 60 ms baseline period over all trials in a given session) does not depend on training (p = 0.49, two-sided t test). G, RF size changes with learning. Mean ± SEM RF diameter for early- and late-training animals are significantly different (p = 0.009, two-sided t test).
The contour plots suggest that the locations of the IR-RF peaks are quite variable from neuron to neuron. However, they are not randomly distributed. For the 179 S1 units recorded under the normal mapping, the distribution of IR-RFs peaks tend to cluster near where the IR stimuli clustered, in the anterior/medial space of the IR stimulus population vectors (Fig. 6B, red ellipse, shows the 95% confidence ellipse for the mean of the distribution of IR-RF centers). For comparison, Figure 6C plots the distribution of all stimulus population vectors presented to all animals that were recorded. Quantitatively, the 2D correlation between the IR-RF centers and the actual distribution of IR stimuli was 0.44, a significant correlation (p < 0.001, F test). Altogether, these data suggest that the overall distribution of neuronal IR-RFs tended to reproduce the spatial distribution of stimulus population vectors used, though with a slight preference for larger stimuli (i.e., closer to the borders of the diamond) on the mediolateral axis.
To test the hypothesis that response preferences in S1 neurons tend to match those of the stimuli delivered, we then compared the IR-RFs for three animals in which the mapping between front and rear channels was reversed (104 units). If neurons tend to naturally prefer activation of the anterior channels (Fig. 1B), then we would expect the IR-RF centers to be localized in the posterior portion of the population vector space when the front and back channels are reversed. However, this is not what we observed. Instead, the IR-RFs continued to favor the front stimuli, with the mean IR-RF center located at <−126.3, 205.4>, and a significant 2D correlation between IR-RF centers and stimulus distribution (2D correlation coefficient, 0.20; p = 0.003, F test). This confirms the hypothesis that response profiles, as summarized in the IR-RFs, tend to match the distribution of stimuli actually delivered during the task.
There was not a simple relationship between stimulating electrode location (or even hemisphere) and RF center. Neurons did not simply respond maximally to localized stimulation. Neurons tended to respond most strongly to stimulation delivered in two locations (Fig. 5, inset), often between hemispheres. Of 179 S1 units recorded with the normal mapping, 65% (117 of 179 S1 units) had receptive field centers located in a region in which at least two microstimulators were active (Fig. 6D,E). Of those with a preference for stimulation at two or more locations, 58% (68 of 117 S1 units) had a preference for stimulation across hemispheres.
We next examined the effects of learning on RF magnitude and diameter. Figure 6F displays the IR-RF peak magnitude when the behavioral performance of the animals was fairly poor (<40% correct) versus when rats reached excellent behavioral performance (>90% correct). This did not change with learning (p = 0.91, t test), suggesting that S1 responds with robust responses very early in training, which is what we observed informally even on the first day that animals were training on the IR task. Figure 6G depicts the IR-RF diameter as a function of behavioral performance. The IR-RF diameter is a measure of the area enclosed by the contour line at 75% of the maximum value of the IR-RF (see Experimental procedures). This analysis revealed that the IR-RF width actually increased with learning (p = 0.009, t test).
Population discrimination of 75 ms stimulus pulses
Next, we addressed the question of how well S1 neuronal firing patterns could predict the stimulus presented to the animal in a given 75 ms epoch. To do this, we applied naive Bayesian classifiers. Because many stimulus vectors do not occur often enough to build a reliable classifier, we narrowed our classification problem to the eight most frequent IR stimulation patterns in each session (a 3 bit classification task). We then ran the classifier on different numbers of S1 neurons, from 1 to 14 units at a time, depending on how many we were able to record simultaneously. Figure 7A displays the mean performance of the classifier as a function of the number of neurons used to train and test the classifier: a neuron-dropping curve (Wessberg et al., 2000; Lebedev, 2014). The red line depicts chance level. As one can see, the percentage of trials correctly classified by the Bayesian algorithm rose from 20% (using one unit) up to 52% when the spike counts from 14 units were combined into a single vector.
Discrimination of microstimulation patterns in S1. A, We selected eight of the most frequent population vectors for each session and used a Naive Bayes' classifier to predict the stimulus based on neuronal activity in S1. Plot shows the percentage of population vectors classified correctly as a function of the number of units used to predict the stimulus (mean ± SEM, N = 20 sessions, 299 multiunits total). The red line is chance (12.5%). Note that this represents a differential response to a single 75 ms stimulation bout (e.g., stimulus T3 in Fig. 4B). On average, there are 11.3 ± 5.1 stimuli/trial (mean ± SD). B, Same data as in A, but with discrimination separated by behavioral performance, as in Figure 6.
When we broke our analysis to look into high- and low-performing sessions, as described in Figure 6, we observed that neural ensemble discrimination of single-trial IR stimuli was significantly better in highly trained animals (Fig. 7B; p = 0.048, two-sample Kolmogorov–Smirnov test). This suggests an improvement in neuronal discrimination power as a function of learning.
S1 responses to uninformative microstimulation patterns
Electrical stimulation of the brain, outside of any task context, clearly produces significant responses, and can even induce drastic plasticity (Nudo et al., 1990; Recanzone et al., 1992; Maldonado and Gerstein, 1996). To examine whether the responses we observed depended on the microstimulation patterns providing information to be used to perform the IR discrimination task, we delivered identical microstimulation patterns in two control animals, but in a way that was unrelated to any task (see Experimental methods). Briefly, we pulled stimulus sequences from rats that actually performed the task, and delivered them at random times to animals that performed a normal visual version of the task (no IR light), such that microstimulation provided no information about the stimulus in each port. This allowed us to reproduce the general context of stimulation, but remove its task relevance.
In both animals, after 2 weeks of training with task-irrelevant microstimulation, S1 sensory-evoked responses were drastically suppressed. Figure 8A shows an example from one animal in which we recorded 14 S1 units simultaneously. The mean response to the first three and last three stimuli is shown in Figure 8B, showing the gross suppression effect (Fig. 8B, top) compared with the normal animals (Fig. 8B, bottom). Overall, the mean ± SEM of the maximum response to all stimuli was 1.4 ± 0.13 when the stimulation was uninformative, and 17.4 ± 1.6 when informative, showing a significant difference (p < 0.001; Fig. 8C). Similarly, the mean response magnitude as a function of the number of stimulating channels activated was effectively a mirror image of the case with informative stimulation, with values reflected across the x-axis (Fig. 8D).
Uninformative stimuli produce inhibitory responses. A, PSTH in all neurons recorded in an animal receiving noninformative microstimulation sequences. Specifically, we delivered microstimulation sequences pulled from animals performing the actual IR discrimination task, but delivered at random times, to an animal doing an unrelated visual task. B, Mean response magnitude to the first three and last three stimuli in two animals, six sessions, 78 units. Red bars, Response when ICMS is not task relevant; black bars, mean response when ICMS is used for IR discrimination. Note the inhibitory response to uninformative ICMS. C, Mean peak response magnitude in all neurons in which ICMS was not task relevant vs when it was task relevant. D, Mean ± SEM response magnitude (z-score) as a function of number of units active for the task-irrelevant (red) and task-relevant (black) ICMS. Again, note the difference in magnitude and the sign of the response. E, Surprisingly, the inhibitory responses carried just as much information about stimulus identity. The discrimination performance as a function of the number of units is shown for animals stimulated during the IR task (black), outside the task context (red), and with stimulus labels shuffled (gray). This suggests that information about microstimulation patterns is present in latent form, ready to be exploited by the brain when it becomes task relevant.
We also examined the ability of the neuronal responses to discriminate microstimulation patterns, expecting it to be significantly diminished compared with the case when such patterns were informative about events in the environment of the animal. Surprisingly, these inhibitory responses carried almost as much information about microstimulation patterns (Fig. 8E; see Discussion).
S1 neuronal firing responses to whisker deflections in rats implanted with IR neuroprosthesis
We observed strong rapid S1 neuronal sensory-evoked responses to whisker deflections in animals trained on the IR discrimination task. Figure 9A shows the carbon monoxide-stained section through both hemispheres for one animal that was highly trained in the task (performance consistently above 95% correct) to show the location of the recording electrodes. Figure 9B displays the responses of 15 S1 units (multiunit activity), in the same animal, to deflections of the right whiskers using an air puff. We observed significant responses in all S1 units in the three rats (N = 36 units; Griffin et al., 2011). The S1 neuronal response latency was 10.8 ± 0.38 ms, and the mean response duration was 92 ± 12.5 ms. The response magnitudes were quite large: the mean number of spikes above baseline was 728 ± 91 total spikes for multiunit activity and 77 ± 21 total spikes from 15 single units in the same animals. The mean response to all whisker deflections, over all 36 multiunit recordings, is shown in Figure 9C; clear onset and offset responses can be observed. These results suggest that S1 has not been hijacked by the IR neuroprosthetic (Griffin et al., 2011). Instead, S1 seemed to have incorporated an ability to represent both stimuli simultaneously into a multimodal map of the tactile and IR environment.
S1 neurons that respond to ICMS are still responsive to whisker deflections. A, Flattened cortical sections through both S1 hemispheres in one animal show the location of electrodes. The asterisks mark the electrode locations. B, Multiunit response to whisker deflections is robust for all units. Figure shows PSTH in units from the same rat depicted in A, after training on the IR discrimination task, upon deflection of the right whiskers with a 200 ms air puff delivered as the rat sat in the behavioral chamber. C, Mean multiunit response to whisker deflection, in all 36 units recorded in three animals. Bin width, 1 ms.
Discussion
The adult mammalian cortex is able to overcome the biological constraints imposed by the native peripheral sensory transducers of an organism, absorbing a spatially distributed representation of a completely novel sensory modality. Adult rats quickly adapted their behavior to use a panoramic IR neuroprosthesis, attaining near perfect performance in a sensory discrimination task involving the new sensory modality (Fig. 2). The cortical neuroprosthesis used in this study is, as far as we can tell, the first device to project a panoramic topographically distributed projection of the subject's surround to both hemispheres of a primary cortical area. Based on the ablation experiments (Fig. 2H), it is clear that rats used all four information channels, though none are indispensable. Instead, performance dropped gradually with each IR sensor inactivated, but even when only one sensor was active, performance was still significantly above chance (Fig. 2H).
Note that the use of IR sources was largely arbitrary. We could have used any information source that bypassed the natural transduction pathways of the rats, as long as a portable sensor was available (e.g., magnetic fields; Nagel et al., 2005; Norimoto and Ikegaya, 2015).
In the 1960s, Bach-y-Rita et al. (1969) pioneered sensory substitution experiments in which continuous streams of visual information were delivered mechanically to the skin on the back of blind human subjects, using actuators laid out to preserve the topographic organization of the original visual stimulus. While our rats needed just a few days to show excellent performance with the prosthesis (Fig. 2), in humans there is a great deal of variance in the learning curves for peripheral sensory substitution devices. The time taken to get used to different peripheral devices ranges from 30 h to months (Nagel et al., 2005; Amedi et al., 2007; Kim and Zatorre, 2008; Kärcher, 2012; Kaspar et al., 2014). Even compared with the studies with the shortest learning times (30–40 h), our rats performed as well or better after 3 h (Amedi et al., 2007; Kim and Zatorre, 2008). Hence, we hypothesize that people who one day take advantage of sensory neuroprosthetic systems that deliver information directly to the CNS—particularly primary sensory areas—will learn to use the new information faster than those using feedback delivered to the periphery. Current motor neuroprostheses typically record a motor control signal in the brain and send this to a prosthetic limb (Chapin et al., 1999; Carmena et al., 2003; Nicolelis, 2011; Ifft et al., 2013). Using the technology described here, the brain could be placed in a continuous two-way communication with information from sensors in prosthetic limbs (O'Doherty et al., 2012). Our data suggest that when designing prosthetic systems for primary sensory areas, it would be prudent to exploit preexisting topographic maps, as rats learned faster when stimulating channels were aligned with the somatotopic map of the barrel field (Fig. 3I). One explanation of this might be that the brain already contains the downstream connectivity pattern required to extract spatial information from the topographic map in S1, and this makes the natural mapping easier to acquire (Diamond et al., 1999; Harris et al., 1999).
Previous work has shown that the barrel cortex integrates information across hemispheres to perform simple spatial discrimination tasks (Krupa et al., 2001; Shuler et al., 2001). Our results show that interhemispheric stimulation patterns can be readily discriminated using a topographically distributed multisite ICMS in a very short time.
Neuronal coding of IR stimuli
After training with the IR neuroprosthesis, S1 neurons simultaneously encoded both tactile and IR stimuli in a synthetic multimodal map (Fig. 9). Future experiments will be needed to determine the effects of this new IR modality on the ability of the animals to perform tactile discrimination tasks using their whiskers (Ni and Maunsell, 2010).
One surprising finding was that inactivation of rear-facing channels had a more deleterious effect on IR discrimination than inactivating the forward-facing IR sensors. The front channels were the dominant source of inputs to the system for the duration of most trials (Fig. 4F), so why did animals seem to preferentially use the rear channels? One possibility is that the rats use heightened activity in the back channels at the end of a trial as an indicator that they are getting very close to the correct port. Indeed, Figure 10 shows that the mean output of the rear voltage sensors actually increases when they are very close to the correct port.
Voltage traces from four IR sensors. Representative mean ± SEM sensor output as a function of time during trial. Time 0 is the time at which the rat selected a port. Note that ∼1.5 s before the end of the trial, the two front sensors rapidly saturate as the animal runs toward the port, while the signals from the rear sensors drop. However, when the animal gets very close to the port, the IR source is broad enough to actually stimulate the rear IR sensors, as you can see with the final peak close the time of the poke. See Discussion.
Neurobehavioral plasticity
There were multiple types of plasticity exhibited here. Clearly, animals had to learn to use the new information source projected to S1. This demonstrates that, throughout the life of the animal, the brain continues to monitor the statistical structure of its inputs. Even at a behavioral level, animals made subtle adjustments to their behavior to more closely align the sensors on their head with the IR sources in their environment (Fig. 2E,F). It will be crucial, in a future study, to track behavior at finer spatiotemporal scales, and fully quantify the relationships among stimulation, behavior, and the neuronal response in S1.
At a finer-grained level, our data suggest that responses in the adult cortex continuously adapt so that they will match the structure of those inputs. This was suggested by our original recording data, in which RF centers matched the general distribution of stimuli (Fig. 6). This was confirmed in an independent set of experiments in which IR-RF centers continued to match the input distribution, even when the front and rear channels were swapped. These results are similar to what is observed in development (Sengpiel et al., 1999).
Future experiments should probe just how far this plasticity can be pushed. For instance, it would be helpful to deliver predefined test pulses in awake and anesthetized animals to determine the degree to which responses to those test pulses indeed change over time.
While the brains of rats were quite capable of adopting new behavioral strategies to harvest information from their IR environment, the remapping experiments examined the ability of animals to change that strategy when pushed into unusual regimes it would not normally experience. Thereby, we follow experiments like those using the retinal inversion lenses by Stratton (1896), the rewiring studies of Sur et al. (1988), frogs with rotated eyes by Udin (1983), and the work on the effects of adding a third eye to the frog by Katz and Constantine-Paton (1988). We are not aware of such inversion/remapping experiments being performed in previous sensory substitution systems in humans. While rats initially performed poorly after a remapping was applied (Fig. 3A), they quickly adapted when the remapping was relatively simple (front-back or left-right switches). However, when the topography was more complex (the diagonal remapping), they needed the help of visual cues to relearn the new mapping (Fig. 3B,D). This illustrates the sensitivity of the brain to pre-existing topographic maps, even artificial maps, suggesting that that it cannot learn arbitrary input/output mappings.
Interestingly, once animals learned one remapping, they learned subsequent remappings significantly faster (Fig. 3G). This suggests that the remapping experiments are triggering a central metaplasticity mechanism, such that animals literally learned to learn the new mappings. Such a boost in learning rate after being exposed to demanding or enriched environments and tasks was suggested by Hebb (1949). While metaplasticity may involve the modulation of molecular-level synaptic plasticity mechanisms (Abraham and Bear, 1996; Rebola et al., 2011), there are likely also higher-level cognitive mechanisms at work, such as changes in the expectation of the animal about the task. With the present rodent model, we can more thoroughly explore these options, for instance by blocking NMDA receptors during remapping experiments.
One surprising effect was the influence of the task relevance of microstimulation on responses in S1 (Fig. 8). We delivered uninformative microstimulation patterns in rats performing an unrelated visual task. We expected similar response properties in both groups of animals, as it seems that passive stimulation is sufficient to induce significant response plasticity (Nudo et al., 1990; Recanzone et al., 1992; Maldonado and Gerstein, 1996). Instead, we saw drastically suppressed responses in the majority of units examined (Fig. 8). This complements previous observations that responses in primary sensory areas are strongly dependent on task context (Fanselow and Nicolelis, 1999; Fanselow et al., 2001; Nicolelis and Fanselow, 2002; Niell and Stryker, 2010; Fu et al., 2014). Our results suggest that responses depend not just on the statistical structure of the inputs, but also on their relevance. The causes of the suppressed activity could be a combination of attentional effects, differences in movement patterns when the information is not relevant, or other top-down corticocortical effects on S1 processing (Pais-Vieira et al., 2013).
Curiously, these suppressed responses also carried significant information about the stimulation patterns delivered to the brain (Fig. 8E). Hence, while the S1 neuronal responses to uninformative stimuli were suppressed, the information remained latent in patterns of inhibitory activity. We hypothesize that the cortical response profile would become excitatory when the patterns of stimulation become informative about events significant to the animal.
Overall, this study highlights the remarkable plasticity of the adult mammalian brain. The brain seems to remain exquisitely tuned to different information sources well past any classic critical period in development. With the addition of more information channels, the ability of adult rats to exploit the information only improved (Fig. 2). Further, S1 neurons showed clear preferences for those stimuli statistically most likely to be delivered during the task, suggesting that the cortex continuously tunes its responses to the statistical structure of its inputs. However, the brain did not indiscriminately respond to the most likely stimuli. Rather, S1 responses were actually suppressed when statistically probable stimuli were not informative about task-relevant features in the environment of the animal. That is, in addition to activity-dependent plasticity mechanisms, powerful relevance-sensitive mechanisms strongly sculpt cortical activity at a given time.
Footnotes
This work was funded by the National Institutes of Health (NIH) Director's Pioneer Award DP1-MH-099903, and NIH Grant R01-DE-011451 to M.A.L.N. The content is solely the responsibility of the authors and does not necessarily represent the official views of the Office of the NIH Director or the NIH. We thank G. Lehew and J. Meloy for help building the experimental setup and electrodes; T. Hanson, A. Fuller, and J. O'Doherty for help setting up and troubleshooting stimulators; S. Halkiotis for editing and preparing the manuscript; M. Viera for advice at multiple stages of the experiments; Manu Raghavan for statistics advice; Derek Silva Moreira for experimental help; A. Graneiro for help troubleshooting the setup and running experiments; D. Guggenmos for advice with the histology; anad J. Pedowitz for help in training animals.
The authors declare no competing financial interests.
- Correspondence should be addressed to Dr. Miguel A. Nicolelis, Box 3209, Department of Neurobiology, Duke University, Durham, NC 27710. nicoleli{at}neuro.duke.edu
