Listening to the Data: Computational Approaches to Addiction and Learning

Classic inferential approaches

Classical inferential statistics have been traditionally used for hypothesis testing (e.g., χ², t tests, ANOVAs) and/or to analyze relationships between independent variables and one or more dependent variables (e.g., regression analyses). These methods are powerful, rigorous, and reliable when properly used, and they have advanced our understanding of the neurobiology of substance use. However, there are some limitations to their use. Many of these methods assume data normality and independence, which can be difficult to obtain in behavioral and cognitive neuroscience (Allua and Thompson, 2009). Classic inferential approaches are sensitive to outliers, and violations of normality and independence assumptions create misleading and biased results (Amon and Holden, 2021). When outlier data are removed to meet assumptions, valuable and representational data may be removed with it. Subjects of greatest interest to addiction research may lie in the extremes of resultant data (e.g., are higher drug-seekers). Additionally, classic inferential statistics are challenged when multiple hypotheses are tested at the same time, leading to increased Type 1 errors (false positives) (P. F. Smith, 2017), and corrections for multiplicity on large datasets can be problematic, leading to overly limiting or powerful comparisons (Berry, 2007). Moreover, given that most hypotheses can only be tested in terms of “X statistically different from Y,” these approaches have difficulty capturing complex relationships and interactions of pathogenic components that characterize psychiatric disorders, including SUDs.

Computational approaches can address some of these challenges. They can facilitate the processing of large and diverse datasets and provide model-based hypotheses to guide data collection and analysis. They can be used to model behavioral, cognitive, and neural processes; uncover new variables; detect patterns; and integrate multiple data modalities. We suggest that there are certain contexts in which computational methods may yield more useful and valid results, and that these approaches can supplement findings from classic methods to explore deeper meaning in extracted data. In the following sections, we will expand on a few notable examples, including the use of reinforcement learning algorithms, Bayesian inference, evidence accumulation models, multidimensional scaling, and principal component regression.

Algorithmic models

Here, we loosely use the term “algorithmic models” to describe mathematical frameworks by which complex relationships between neurobehavioral variables are formally stated in a set of rules describing the expected impact that a variable's state change has on its related variables. Such algorithmic models have been extensively used in addiction neuroscience (Redish et al., 2008; Mollick and Kober, 2020; R. Smith et al., 2021; Kato et al., 2023). These approaches allow us to disentangle and measure complex, dynamic behavioral constructs that are otherwise ambiguous and hard to quantify. Fitting behavioral (or neural) data to algorithms allows for testing of specific quantitative hypotheses about the structure of decision-making or cognitive processes and their plasticity (e.g., because of learning or neuro-modulation), mapping precise psychological constructs to experimental results. These algorithms provide generative value, informing future work and analyses, and aiding the interpretation of extant data (Tolomeo et al., 2021).

Algorithms based on reinforcement learning have already advanced our understanding of maladaptive decision-making, which is a hallmark of SUDs (Redish, 2004). Within this framework, goal-directed decision-making requires the selection and execution of one behavior in response to stimuli, followed by appraisal of that choice, and adjustment of future decisions given past outcomes to obtain the optimal outcome. Computationally, goal-directed behavior may be best represented by model-based reinforcement learning frameworks, while compulsive behavior may be best represented by model-free frameworks (Dayan and Niv, 2008; Daw et al., 2011; Montague et al., 2012; Dolan and Dayan, 2013; Voon et al., 2017). This proposed model-based/model-free dichotomy is consistent with the theory proposed by Everitt and Robbins (2016) that the transition from voluntary, recreational (goal-directed) drug use to compulsive, habitual use underlies the maladaptive decision-making pattern that characterizes SUD.

In a recent publication, Costa et al. (2023) created a novel model-based reinforcement learning algorithm to characterize the role of the lateral orbitofrontal cortex (lOFC) in learning, in male rats. The authors inhibited the lOFC during the initial training of two outcome-specific associations, and later tested whether a cognitive map was properly formed in a reinforcer devaluation probe. Classical inferential analyses suggested that lOFC inactivation impaired model-based learning but revealed little about the underlying cognitive mechanisms. Several reinforcement learning models were then fit to individual behavioral responses, and this showed that it was unlikely that the lOFC was responsible for deploying associative representations during model-based task performance (one of the leading hypotheses in the field), or for creating cognitive maps per se (a competing interpretation). Instead, reinforcement learning algorithms revealed a circumscribed role of the lOFC in creating outcome-specific maps, essentially determining the resolution of learned associations. This detailed interpretation was only possible by comparing the fits and predictions of precise algorithmic models. Critically, lOFC dysfunction and the inability to properly form cognitive maps are thought to be a key feature of SUDs (Schoenbaum et al., 2016); and being able to dissociate subtle cognitive variables, as was done in this study, is an important step toward understanding how changes in brain function can lead to behavioral pathology.

Bayesian approaches are widely used in computational psychiatry, both in the context of modeling human and animal cognition (e.g., in estimations of precision/uncertainty in perception and decision-making) (Diaconescu et al., 2014, 2017; Ma and Jazayeri, 2014; Stephan and Mathys, 2014) and in Bayesian data analysis of computational methods (Lee, 2011; Ahn et al., 2014; Vandekerckhove et al., 2018; Lee et al., 2019). However, Bayesian inference approaches are relatively new in SUD data analysis, with distinct strengths compared with other methods (R. Smith et al., 2021; Kato et al., 2023). Frequentist (non-Bayesian) models underperform when accounting for enhanced habitual responding in drug users (Lim et al., 2019), or when reward-seeking behaviors are not cue-induced (Kato et al., 2023). Bayesian inference models, in comparison, overcome these limitations by computing an agent's states, actions, and outcomes as probability distributions (equivalent to priors and posteriors in Bayes' theorem). Priors update to posteriors based on a prediction error signal, which is hypothesized to be encoded by dopamine phasic bursts, representing the discrepancy between one's beliefs and actual observations (Friston et al., 2012). Expanding on this line of thought, Kato et al. (2023) proposed that Bayesian inference models cannot only extend the explanatory potential of reinforcement learning theories, but also incorporate behavioral addictions (i.e., gambling), which may not depend so heavily on the reward/cost function. In contrast, reinforcement learning reward prediction error-based theories heavily pivot on the reward value because they assume that decision-making alterations are caused by dopamine-related, drug-induced alterations determining the reward/cost function instigating motivated behavior.

Evidence accumulation models, such as the drift diffusion model (DDM), are another category of algorithmic models that allow for analysis of the latent cognitive processes involved in decision-making (Ratcliff, 1978; Myers et al., 2022). These models can account for choices as well as shapes and locations of reaction time data simultaneously, a feature that allows them to successfully explain complex behavioral data from decision-making tasks. Initially developed to explain simple, two-choice perceptual decisions in humans (Ratcliff, 1978), the DDM deconstructs the decision-making process into key contributing factors: the decision starting point, the information accumulation rate toward an option, thresholds that determine the required quantity of information before decision-making, and the contribution of nondecision elements (e.g., sensory and motor processing) to the total response time. Use of the DDM can reveal differences between groups in underlying cognitive processes, despite behavioral profiles looking similar when analyzed using traditional methods. DDMs have been successful in modeling decision-making across numerous domains (both perceptual and value-based), including in aging, child development, clinical populations, and animal species (Ratcliff, 1978; Myers et al., 2022). Further, correlates of the processes described by DDMs have been validated through neural and brain activity measures (e.g., electrophysiology, EEG, fMRI) (Gupta et al., 2021).

DDMs have been used to model the psychological and neurobiological processes in substance-related and addictive disorders in humans. In comparing ex-nicotine smokers and current daily smokers on a value-based decision-making task, drift diffusion modeling revealed that ex-smokers were more cautious in making value-based decisions about tobacco-related cues than current daily smokers (Copeland et al., 2023). To investigate the effect of binge-drinking intoxication and hangover on response selection, DDM and EEG were combined, revealing that acute intoxication decreases information accumulation rates and shortens nondecision information encoding times which were related to specific changes in EEG event-related potentials (Stock et al., 2017). In gambling disorder, the use of DDM has linked reduced decision thresholds (less cautious responding) with increased risky choice and active gambling (Peters and D'Esposito, 2020; Peters et al., 2020; Bruder et al., 2021).

Preclinical behavioral tasks commonly incorporate two-choice decision-making components, which yield data that are ideal for DDM analysis; however, DDMs are underutilized in animal studies. DDM parameters represent common cognitive processes (not necessarily task- or species-dependent) that are comparable between paradigms. By using all choice data and the full shape of the reaction time distributions for these choices, rather than traditional metrics such as the mean and SD in classical inferential approaches, DDMs allow for the uncovering of underlying decision-making processes that drive behavior. DDMs can also be incorporated in reinforcement learning models as the decision rule for action selection in place of softmax (Fontanesi et al., 2019), which allows for concurrent investigation of reward learning and decision-making. Increased application of DDMs to rodent behavioral datasets may provide novel insight into the underlying cognitive processes and neural substrates that contribute to maladaptive decision-making in addictions as well as improve interpretability and relevance of findings from animal models of addiction to humans (Hales et al., 2023).

An implementation of the DDM that uses hierarchical Bayesian methods to estimate model parameters, the hierarchical Bayesian estimation of the drift diffusion model (HDDM) (Wiecki et al., 2013), can simultaneously estimate individual values and group distributions while providing more precise parameter estimates. The HDDM can be used to investigate how trial-by-trial measurements, such as outcomes from fMRI, influence decision-making processes. In preclinical settings, HDDM has the potential to better model latent cognitive variables at play during multiple-choice paradigms where an animal must decide whether to seek a drug or a nondrug reward (food, social interaction, etc.) (Ahmed et al., 2013). HDDM can readily detect dependencies of latent parameters on different brain measures (Wiecki et al., 2013), such as Ca²⁺ influx dynamics, neuronal firing, or neurotransmitter release, offering an unparalleled opportunity to reveal novel abnormal cognitive processes underlying excessive drug reinforcement and subsequent displacement of alternative rewards. As models evolve in sophistication and require inclusion of numerous parameters, integration with other types of methodologies, such as dimensionality reduction (discussed below), can be useful to help identify the key features explaining behavior.

Dimensionality reduction approaches and machine learning

The neurobiology of SUD is inherently complex. SUDs are commonly comorbid with other psychiatric disorders (Udo and Grilo, 2019; National Institute on Drug Abuse, 2020). Additional consideration of genetic factors, environmental influences, polysubstance use, sex differences, and individual variability yields intricate datasets that are challenging to interpret. Dimensionality reduction methods are valuable tools to reduce complexity while retaining essential information. Two notable approaches to analyze this type of data, principal component analysis (PCA) and MDS, are unsupervised machine learning algorithms that reduce data dimensionality and noise, prevent overfitting, and increase interpretability. As with reinforcement learning algorithms, the use of dimensionality reduction is not new and has been previously applied in addiction research (Konu et al., 2001; Kramer et al., 2010; Maremmani et al., 2017; Hoffmeister et al., 2019; Tan et al., 2020; Dunn et al., 2023). While similar in goal, the methods and approaches differ. MDS focuses on the correlational relationships between variables and maps them onto two-dimensional space for visualization, while PCA transforms such relationships into new variables, called principal components, that capture the covariance in data (Hout et al., 2013; Jolliffe and Cadima, 2016). The resulting principal components can represent a hidden variable, not directly measurable in the experimental design, but patent in the set of interrelationships found between the observations obtained. In this way, these methods are valuable for exploratory and confirmatory data analysis, pattern recognition, and to elucidate relationships that were not yet apparent from the use of classical approaches in addiction research.

Principal components identified by PCA can be applied as new independent variables in classical inferential analyses, such as linear regression. Linear regression is sensitive to multicollinearity, departures from normality, and overfitting of multidimensional data. Principal component regression (PCR) is a two-step process by which new independent variables are first uncovered by PCA, and then fit as regressors in a subsequent model. By first reducing dimensionality through PCA, this approach alleviates issues of multicollinearity and overfitting. Further, it transforms the data into experimental outcomes predicted by underlying constructs or processes, revealing relationships not directly observed through experimental outcome measurements.

In a few notable examples from recent years, PCA has aided in the characterization of precipitated withdrawal symptoms following opioid antagonist or partial agonist administration (Dunn et al., 2023) and helped identify common personality traits among drug users for comorbidity theory development (Maremmani et al., 2017). This method helped classify features of alcohol craving unique to those with alcohol use disorder (Kramer et al., 2010), discerned factors predictive of alcohol use disorder development (Hoffmeister et al., 2019), and identified genetic-behavioral endophenotypes underlying vulnerability to excessive drug-seeking (Flagel et al., 2016; Slosky et al., 2022). PCA has also advanced our understanding of addiction neurobiology in revealing the protective effect of luteolin (a flavonoid) reducing meth-induced neurotoxicity is because of intricate changes in striatal PI3K/AKT intracellular cascades in male rats (Tan et al., 2020).

Recently, Luján et al. (2023) used PCR to elucidate valuable information from NAc fiber photometry readouts during cocaine self-administration in mice. Cocaine-evoked dopamine transients were recorded from male and female mice during every drug interaction throughout self-administration, extinction, and cue-induced reinstatement. Both classical inferential analysis (Pearson's correlations) and PCR identified a significant relationship between cue-evoked dopamine transients and reinstatement. However, only PCR detected a relationship between drug-evoked dopamine transients, characterized by increases in dopamine fluorescence following intravenous drug delivery, and relapse incidence, because of the high dimensionality of the resultant data. Individual dopamine measurements were first reduced by PCA to three low-dimensional representations of drug-evoked dopamine reactivity, accounting for 75% of all dopaminergic variance. Then, the principal components were applied as regressors to predict cue-induced reinstatement of cocaine-seeking. PCR revealed that the low-dimensional representations of drug-evoked dopaminergic reactivity accurately predicted reinstatement, accounting for 73% of the observed variance (increased to 82% when considering sex). In this case, a computational approach was better fit for the complex data output than classical approaches. In the same study, variations in accumbal dopamine release were examined between male and female mice. A conventional inferential approach (averaged fluorescence amplitude Student's t tests) identified numerous sex differences, seemingly incongruent with the comparable levels of relapse behavior in both sexes. By revisiting the low-dimensional representation of dopamine responses from PCA, a shared pattern of dopamine release was found, reconciling the neural and behavioral findings. This highlights the potential utility of PCA in revealing hidden patterns and associations within the data, which may not be immediately apparent using traditional inferential analyses alone.

Challenges to data analysis are presented when a dataset contains a large number of predictors relative to a dependent outcome with limited data points, or a low sample size. The least absolute shrinkage and selection operator–principal component regression (LASSO-PCR) algorithm was developed to overcome this challenge. LASSO-PCR is a multistep machine learning approach similar to the PCR method mentioned above. First, PCA reduces dimensionality by identifying principal components. The principal components are then fit as regressors to a linear regression model with a LASSO algorithm, which refines the prediction, aiding in variable selection (Tibshirani, 1996). Using this method, Koban et al. (2023) found that whole-brain activity evoked by visual cues of drugs in adults with SUDs and their matched controls can predict self-reported craving (p < 0.0002) as well as discriminate between drug users and nonusers with a high degree of cross-validated accuracy (82%). After performing dimensionality reduction of brain-wide fMRI using data and identifying brain networks characterized by high covariation of voxels (PCR), the LASSO algorithm implemented a refinement to the prediction (in this case, reported feelings of craving) by penalizing the contribution of less relevant principal components. The analysis arising from LASSO-PCR permitted the determination of a Neurobiological Craving Signature, encompassing cue-evoked brain activity patterns from the ventromedial PFC, ventral striatum, supplementary motor area, and anterior midcingulate cortex, as a reliable predictor of self-reported drug craving and classifier of drug use status (Koban et al., 2023). This work indicates the potential for the identification of objective diagnostic, prognostic, and predictive neural markers of drug craving and relapse in humans (Food & Drug Administration-National Institutes of Health Biomarker Working Group, 2016). To the best of our knowledge, we are unaware of other applications of this method in SUD research.

Compared with PCR approaches, MDS is less widely used in addiction neuroscience. As mentioned above, MDS permits the visualization of interrelationships of multiple variables of interest in two-dimensional space. Commonly, an additional unsupervised machine learning algorithm, K-means clustering, is used to segment these visualized data into groups based on similarity, with “K” representing the number of desired clusters. MDS used with K-means clustering has revealed previously unseen relationships between pain sensitivity and opioid reward in rodents (Brice-Tutt et al., 2023).

Additionally, MDS has been used to uncover complex relationships between sex, alcohol and oxycodone co-use, and neural activation in rats (Wilkinson et al., 2023). In this model, male and female rats self-administered oxycodone for 3 h followed by 6 h access to two-bottle choice alcohol/water. Economic demand for oxycodone was assessed, followed by reestablishment of baseline consumption of one (oxycodone or alcohol) or both (oxycodone+alcohol) drugs, extinction, and cue-induced reinstatement testing. Brains were processed and analyzed for mesolimbic c-fos mRNA expression immediately following reinstatement. Classic inferential statistics (ANOVA) revealed no differences in oxycodone demand parameters or c-fos expression between oxycodone-only and alcohol+oxycodone-consuming male rats. However, MDS with K-clustering revealed dissimilarities in recruited brain regions during cued reinstatement of oxycodone-seeking: c-fos in the BLA, NAc core, and shell, related to reinstatement in oxycodone-only males, while c-fos in the dorsal striatum and prelimbic cortex related to reinstatement in male rats exposed to alcohol+oxycodone polysubstance use (Wilkinson et al., 2023). These findings highlight the utility of such approaches: while there may be no identifiable differences between groups (statistically equivalent in dependent variable measures), the complex interaction of independent variables (drug intake and frequency, brain regions recruited, etc.) may be different between sexes and drug groups.

While many are exposed to substances, there is a wide variation of responses, with only a subset that develop an SUD. Attempting to model this variability in preclinical models, dimensionality reduction and clustering approaches are gaining traction to identify vulnerable subpopulations of animals that model this heterogeneity in clinical populations. Recently, Jadhav et al. (2022) used a combined machine learning-assisted clustering approach based on K-median (for cocaine) and K-means (for alcohol) in combination with an artificial neural networks approach that allowed for reliable classification of vulnerability and resilience to addiction-like behavior in rodents across cohorts. In a separate approach, Allen et al. (2021) combined data- and theory-driven approaches in use of a Bayesian degree-corrected stochastic block model (DCSBM) to identify resilient, intermediate, and susceptible subpopulations to opioid vulnerability.

In summary, unsupervised machine learning algorithms directed at pattern recognition and component analysis are important tools for uncovering patterns related to substance use in animals and humans. Specifically, incorporating such approaches alongside, or in combination with, classic inferential statistics enables the identification of relationships not easily observed with traditional methods alone.

In conclusion, in the field of addiction neuroscience, computational approaches are not yet widely adopted, potentially limiting a comprehensive understanding of the neurobiology underlying SUDs and hindering the identification of effective treatments. Here, we highlighted several computational approaches to data analysis that offer distinct advantages, and that will be featured at our 2023 SFN Mini-Symposium. We reviewed the contexts in which computational approaches yield more valid and translationally relevant results than classic inferential approaches, such as when data dimensionality or multicollinearity is high (MDS and PCA), when datasets have a large number of predictors with a limited number of dependent measures or low sample sizes (LASSO-PCR), and when cognitive mapping, learning, and decision-making processes are involved (reinforcement learning, Bayesian inference, and DDM). These methods have revolutionized our conception of addiction through use of objective, highly predictive neural diagnostic markers (Koban et al., 2023). They have identified drug addiction endophenotypes, valuable for the future of precision treatments based on individual differences in animals (Fiore et al., 2018), and promoted theory development on brain regions associated with the transition from voluntary to compulsive drug use (Lucantonio et al., 2014). It is our hope that the present review will encourage the use of computational methods alongside existing approaches to deepen our understanding of extant data, drive hypothesis and framework development, and enable valuable predictions toward advancing addiction neuroscience. By leveraging the power and utility of computational approaches to reward and addiction neuroscience data, the field can move closer toward novel target identification and treatment development.