Toward an Efficient and Integrative Analysis of Limited-Choice Behavioral Experiments

The structure of experimental behavioral data—some background

The relative character and bounded structure of data resulting from limited-choice tests become apparent when preferences between choices are presented as percentages or ratios, which add up to 100, or unity. They may be harder to spot when preferences are expressed, e.g., as absolute times spent with any one choice, or as counts. Yet as the total time of observation in such tests, or total counts, are limited, times spent with any one choice or counts representing them are also bounded, and hence are indicators of relative preferences rather than free variables (Pennington et al., 2009). A hypothetical numerical example construed to model the standard test for social novelty preference for mice (Nadler et al., 2004) should clarify the consequences of this situation for statistical analyses, and why ANOVA is not appropriate for data of this structure. For this test, animals are placed in the neutral zone of a tripartite cage, from where they can choose to enter two additional partitions (zones), each containing a stimulus mouse: one of these is familiar and the other novel (a stranger) to the test animal. Suppose we want to compare the behavior of two groups of animals, A and B, which, over the observational period of 600 s, partition their times as follows: animals of group A spent, on average 360 s with the stranger and 180 s with the familiar mouse; the remaining time (60 s) they passed in the neutral zone. Animals of group B spent, on average, 120 s with the stranger and 20 s with the familiar mouse, and 460 s in the neutral zone. We do not have to bother about variances in this simplifying example.

To assess social novelty preference, time spent in the neutral zone is not really relevant, and so novelty preference is typically quantified by comparing the times spent in each of the two compartments holding the stimulus mice. Standard parametric testing (including ANOVA) does so by assessing the differences of the times. So we are led to conclude that group A animals have a greater preference for the novel (stranger) mouse (360 − 180 s = 180 s) than group B animals, for which this difference is only 100 s.

While this approach does not explicitly mention times spent in the neutral zone, these directly, if somewhat furtively, enter the analysis: given the overall limited observational period, the time an animal spends in the neutral compartment automatically limits the times it may spend with either the stranger or familiar mouse, and hence the difference of these values.

To avoid this hideous interference, we may recast our question and ask: how does a test animal partition the total time it spends on social contacts—i.e., the time it spends outside the neutral zone—between the stranger and familiar mouse? In our example, group A animals spent a total of 540 s on social contacts, of which they allocated 66.7% (360 s) to the stranger, and 33.3% (180 s) to the familiar mouse. Group B animals spent a total of 140 s on social contacts, 85.7% (120 s) with the stranger, and only 14.3% (20 s) with the familiar mouse. So this ratio-based perspective suggests that group B animals show a greater preference for the stranger than those of group A. Before considering how this may be tested formally, we would like to point out that times spent in the neutral zone again entered the analysis, but this time not as a nuisance parameter, as in the ANOVA-based approach, but rather as a sensible parameter to “normalize” the social contact times of various test animals.

We may take this a step further and ask whether time spent in the neutral zone is not an interesting variable in its own right. In our example, times spent in the neutral zone may just as well be used as differential preferences for the stranger mouse to distinguish group A animals (60 s) from those of group B (460 s), and indeed suggest that group B animals might be loners. Note that this perspective allows the experimenter to extract, from a test set up to assess social novelty preference, a measure of sociability, which is typically assessed in a separate test in which the test animal is presented with a choice between a stimulus mouse and an inanimate object.

This exemplifies the point that data resulting from limited-choice tests are typically multivariate—i.e., constituted by multiple outcome variables, as, in our example, descriptors of social novelty preference and sociability. Consequently, such data call for appropriate multivariate methods. ANOVA (and t tests), though, are strictly univariate techniques (Tabachnick and Fidell, 2006, chapter 2) and cannot account for the data in full (Weltje, 2002; see also Filzmoser et al., 2009).

Making behavioral data from limited-choice tests accessible to analysis

Fortunately, there do exist methods appropriate for the fundamentally compositional structure of data resulting from limited-choice tests that also allow the experimenter to take full advantage of their genuinely multivariate nature. These have been developed largely in fields like geology and chemistry, but also in the social sciences, where compositional data are common (Aitchison and Egozcue, 2005; Pawlowsky-Glahn and Egozcue, 2006; Pawlowsky-Glahn and Buccianti, 2011 and further references contained therein). We may readily tap and adapt these methods for behavioral analysis, since they are conveniently implemented in the freely available R computational environment (van den Boogaart and Tolosana-Delgado, 2008; Hron et al., 2010; Rizzo and Szekely, 2010; R Development Core Team, 2011). Indeed, there is precedence for the application of compositional statistics to biologically motivated questions including animal and human behavioral biology [Solberg et al., 2004; Pierotti et al., 2009; Pennington et al., 2009 (note that the latter two studies use implementations of the method in STATA and Matlab, respectively)].

At the heart of this methodology is a familiar and time-honored approach. That is, the data are transformed to conform to the (distributional) requirements of the statistical procedures one wants to use, such that the restrictions inherent in the original data are removed. The transformation that simultaneously removes the bounds of compositional data and allows the experimenter to tackle their correlation structure, thus making them amenable to standard parametric statistics, is referred to as the log-ratio transform. This has been pioneered and popularized by Aitchison and Egozcue (2005; and the extensive bibliography therein). The rationale and the formalism of this approach, its technical details, and its benefits have been extensively described (for review, see Billheimer et al., 2001; Weltje, 2002; Egozcue et al., 2003; Aitchison and Egozcue, 2005; Pawlowsky-Glahn and Buccianti, 2011).

Of the three variants of the log-ratio transformation commonly used, the isometric log-ratio (ILR) transform yields the most suitably transformed data for standard multivariate techniques (Egozcue et al., 2003; Pawlowsky-Glahn and Egozcue, 2006). The key step of this transform is that original data are systematically related to each other. Several technical descriptions of how to calculate the IRL are available (for review, see Egozcue et al., 2003; Egozcue and Pawlowsky-Glahn, 2005), and it may be readily implemented in standard spreadsheet software or any statistical program that allows basic mathematical operations on the data. Convenient implementations are available, for example, in the R packages “compositions” or “robCompositions” (van den Boogaart and Tolosana-Delgado, 2008; Hron et al., 2010). So, we may focus here on the conceptual advantage that the log-ratio transform and the ILR, in particular, offer. If we subject the three times recorded in our above example for each experimental animal—i.e., those spent with a familiar mouse, a stranger mouse, and in the neutral zone—to ILR transformation, two derived variables are obtained, which, for obvious reasons, are also referred to as contrasts or coordinates. The first relates the times spent with the familiar and the stranger mouse to each other, and this serves as a measure of social novelty preference. The second relates the combined times spent with the stranger and familiar mouse to the time spent in the neutral compartment and may be sensibly interpreted as a measure of the overall tendency to seek social contacts (i.e., sociability). Note that derived variables are built on ratios rather than differences as in standard ANOVA.

In addition to transforming the original data so that they become suitable for standard multivariate analysis, the ILR transform also allows the experimenter to contrast the original data such that it becomes easier to give a meaningful biological interpretation. Actually, how exactly the original variates are contrasted with each other depends on the order they are entered into the ILR transform. There exist several mathematically equivalent and consistent ILR transforms for a given dataset. These differ by the perspective they give us on the data. Significantly, the experimenter may choose which perspective to take and which contrasts to form simply by defining the order in which the original variates are entered into the ILR transform. This choice could be informed by prior knowledge and thus focus attention on those variates of the original data that have the greatest diagnostic relevance. Conversely, we may calculate several ILR transforms for a given dataset and then ask which of these result in the contrasts with the best diagnostic characteristics (however we may define them) and which original variates enter those contrasts. For a formalized procedure to determine how all possible contrasts may be systematically formed, see Egozcue and Pawlowsky-Glahn (2005).

While log-ratio transformation provides a unique tool to make compositional data accessible to standard multivariate analysis and at the same time provides an approach for their sensible biological interpretation, there is one shortfall, of direct practical relevance, that needs to be briefly discussed. That is, log-ratio transformations, including ILR, cannot handle data that contain zero values, since neither division by zero nor logarithms of zero are mathematically defined. At first sight, this looks like a mathematical nuisance, but it actually takes us to a central methodological and biological issue. Simply to exclude samples that contain zero measurements (in our example, these could arise if a test animal would not visit one cage compartment at all) would not only be a waste of data, but might even substantially distort subsequent analyses. So, it seems well advised to ask how zero values originated in the first place. If test subjects do not select one of the choices available during the observational period, is this a defining behavior, or does it reflect some shortcoming of our experimental setup? Could we have observed the subject take this choice if we had extended our observational period appropriately? Both situations result in zero values, and in technical parlance the first is often called a structural zero, whereas the second is a case of censored data, sometimes also referred to as a rounded, or trace, zero.

In a realistic experimental setting, structural zeros and those due to measurement sensitivity may be hard to discern. Although definitive advice cannot be given here, a workable approach may be to handle, in a first step, a limited number of randomly interspersed zero values as due to experimental sensitivity and thus replace them by carefully imputed small values. However, any clustering of zeros should alert the experimenter to the possible existence of structural zeros, defining a behavioral pattern. These should not be replaced, but suggest the use of an alternative model (see third paragraph in the next section). A more extensive discussion of this issue, along with advice and sensible methods for replacing zeros, may be found, for example, in the studies by Fry et al. (2000), Palarea-Albaladejo and Martin-Fernandez (2008), and Hron et al. (2010).

Efficient display and analysis of data from limited-choice tests

To briefly document the conceptual advantages granted by consideration of the specific structure of data resulting from limited-choice tests, and to introduce some concepts about how to proceed after data transformation, we now shall discuss, and reanalyze, some selected data on sociability and social novelty preference originally published by Moy et al. (2007, 2008). These data are publicly available from the Jackson Laboratory Mouse Phenome Database (http://phenome.jax.org/; Moy1 dataset).

As so often, a sensible first step in the analysis is to plot the data. In Figure 1, bar diagrams traditionally used to represent the outcomes of tests for sociability and social novelty preference are shown alongside two diagrams particularly suitable for compositional data formed by three variables. Ternary diagrams (Billheimer et al., 2001) allow visualization of the original data. They are based on equilateral triangles, where each corner (vertex) represents one variable of the composition. That is, in our example, one of the behavioral choices offered. Each experimental subject is represented by a single point within this triangle, whose distance from any one corner reflects the relative preference for the behavior represented by this very corner. A subject avoiding one choice altogether is represented by a point on the line opposite to the vertex representing this choice. ILR-transformed data are efficiently displayed in biplots (Aitchison and Greenacre, 2002), which, in the case of a ternary composition, are simple x–y scatterplots in which the two contrasts [say, “familiar/stranger” and “(familiar, stranger)/neutral” in our example above] are plotted against each other. Both ternary diagrams and biplots give an ad hoc impression of data variability (or spread), and also of how individual variates and contrasts formed by them may relate to each other (Fig. 1). Of course, summary statistics (as the mean or median, and measures of data variability) may also be included in such diagrams.

• Download figure
• Open in new tab
• Download powerpoint

Figure 1.

A–C, Data display to detect potentially extreme behaviors. Traditional summary statistics (A, mean ± SEM) suggest that, in the sociability test, A/J mice partition their time about equally between the neutral compartment and the cage compartments housing the object and the stranger mouse. Plotting the data in a ternary diagram (B) reveals that most animals either avoid the object, or the stranger-side altogether, suggesting that a dual-choice model (neutral vs non-neutral zones) may be more appropriate than the triple-choice model implicit in the classical representation and the ternary plot. This is also suggested by the observation that in the biplot (C), most data fall on an essentially straight line. Note that for numerical reasons, zero values had to be replaced by imputation (see main text) to allow generation of the biplot. Data are from Moy et al. (2007). D–F, Multivariate display and analysis allow the experimenter to use intragroup variability to discern DBA/2J and NOD/LtJ mice. The traditional univariate representation (D; mean ± SEM) does not give any clues that DBA/2J and NOD/LtJ mice might differ in their social novelty-seeking behavior. Although variability appears somewhat more pronounced for DBA/2J mice, this eludes formal statistical detection for univariate statistics calculated for untransformed data. In contrast, plotting the data in a ternary diagram (E) or, following ILR transformation, in a biplot (F), readily draws attention to the much more pronounced behavioral variability of DBA/2J mice (gray triangles) compared with NOD/LtJ mice (black circles), though both have identical mean performances (gray “x” and black cross, respectively). This difference in variability is also detectable using appropriate formal testing (see main text). Data are from Moy et al. (2008). G–I, Multivariate compositional analysis allows the experimenter to use an internal correlation structure of behavioral data to probe for intergroup differences. Synthetic data describing two groups of mice for which times spent in the individual compartments of a three-compartment cage and the extent of data variability are virtually identical, as seen either in the traditional (G, mean ± SEM), the ternary (H), or the biplot display (I). However, the biplot (I) clearly documents the essentially inverse relationship between the familiar/stranger variate and the variate that contrasts times spent with either the familiar or a stranger mouse with that in the neutral (center) zone. This is also clearly visible in the ternary plot (H), which documents that animals of group 2, represented by gray triangles, show a positive correlation between social novelty seeking and leaving the center (which may be interpreted as a measure of overall sociability). For group 1, represented by black circles, increased sociability goes along with an increased tendency to contact the familiar mouse; i.e., there is a negative correlation between sociability and social novelty seeking.

Figure 1A–C documents the sociability of A/J mice reported by Moy et al. (2007). When displayed as means ± SEM (Fig. 1A, which reproduces Fig. 4A in Moy et al., 2007), the data suggest that these animals spent about equal times with the stranger mouse, in the object cage, and in the neutral cage partition. Consistently, testing by ANOVA suggests that A/J mice, on average, do not show a preference for social contacts (i.e., show no sociability). Inspection of the data displayed in a ternary plot (Fig. 1B) reveals how misleading this conclusion can be. Indeed, of 19 A/J mice tested for sociability, 7 did not spend any time in the cage partition holding the stranger mouse, and another 7 did not enter the partition holding the object. One mouse did not enter either. Only four animals actually visited all three cage partitions, and these rare animals also strongly favored one partition. This perspective on the data may suggest that A/J mice comprise two subgroups with distinct sociability. A more parsimonious explanation is that they simply stick to their first random choice and that this is a characteristic for this strain. The somewhat larger variability of the sociability data of A/J mice seen when data are fitted to a three-choice model (be that by conventional ANOVA or following ILR transformation; see Fig 1A–C; and compare variability with that of the other examples shown in Fig 1D–I) would thus hint that this model be inappropriate. Indeed, the occurrence of multiple zero values in the data describing A/J sociability behavior should also prompt the experimenter to consider a binary-choice model, rather than a ternary-choice model (as done here). In fact, this dataset seems to be a clear-cut example of a behavior characterized by structural zeros (see the last two paragraphs of the preceeding section). The ternary display clearly draws attention to this situation.

How to proceed after data transformation and inspection is, of course, primarily dictated by the experimental questions one wants to address. Here, we focus on how to take advantage of the multivariate data structure for the efficient and sensible comparison of two or more sets of data, as is often the focus of interest when assessing, say, behavioral effects of genetic or pharmaceutical manipulations (Blundell et al., 2010; Peça et al., 2011; Walton et al., 2012). A highly efficient test to do so was recently presented by Szekely and Rizzo (2009) and Rizzo and Szekely (2010) (implemented in the R-package “energy”), and we use social novelty preference data recorded with DBA/2J and NOD/LtJ mice (Moy et al., 2008) to exemplify its usefulness. Group averages alone, whether calculated from nontransformed or ILR-transformed data, do not suggest that these two strains might differ in their choices of the three cage partitions (Fig. 1D,E). Although the error bars in the classical representation of nontransformed data (Fig. 1D) suggest that intragroup variability might vary, this is much better documented in the ternary diagram and also in the biplot of ILR-transformed data (Fig. 1E,F). Yet testing by ANOVA (Fig. 1D, data displayed) or MANOVA (and also Anderson's robust variant; Anderson, 2001) would not even hint at this obvious difference, simply because these tests cannot discern data based on differences in variability. MANOVA actually presupposes that variablity be similar among the groups compared. Conversely, the Rizzo–Szekely test clearly allows to distinguish NOD/LtJ and DBA/2J mice based on the variability of their social novelty preference (p = 0.002). Clearly, knowledge about the behavioral diversity within a group may be as informative as any measure of group average to guide the search for potential behavioral determinants.

The Rizzo–Szekely test not only allows for the simultaneous testing of multivariate datasets based on their means and variability, it also allows the experimenter to distinguish sets that differ by how individual variates relate (or correlate) with each other within the groups compared. A hypothetical example devised to convey this concept within a biological context is shown in Figure 1G–I. In contrast to the DBA/2J and NOD/LtJ data shown in Figure 1D–F, those of the two groups of animals shown in Figure 1G–I not only have comparable group means, but also have the same overall variability, as may be directly guessed from the error bars in Figure 1G, and is clearly visible in Figure 1, H and I. Actually, the data were constructed so as to have identical variability. However, as is clearly visible in Figure 1I—but also Figure 1H—the two groups differ in how their preference for either the familiar or stranger mouse (Fig. 1I, y-axis) and the overall tendency to leave the neutral zone (i.e., sociability; Fig. 1I, x-axis) relate to each other. This striking behavioral difference goes completely un-noticed in the classical display (Fig. 1G), and it cannot be detected using ANOVA or MANOVA. However, it is readily revealed by the Rizzo–Szekely test (p = 0.036).

This example underscores the importance, and substantial profit, of selecting a test appropriate to the data structure, and also the critical value of visual inspection of original data in this process. While the Rizzo–Szekely test does not tell us whether data differ due to differences in means, their variability (spread), or the internal correlation between individual variates, it allows the diagnostic use of the latter two parameters. Clearly, the potential to probe group differences beyond mean performance, as provided by the Rizzo–Szekely test (Szekely and Rizzo, 2009; Rizzo and Szekely, 2010), enhances overall test sensitivity and thus contributes to an ethical, and more economic, use of experimental subjects. The graphical display afforded by the ternary diagram and biplot should help to decide how to proceed once a significant result is obtained with the Rizzo–Szekely procedure. This may include applying a formal test for differences in scatter, or fitting (linear) models to describe the correlations of individual variates. Here, sensitive model choice may provide substantial insight into the behavioral difference diagnosed.

Conclusions and perspectives

Recognition and proper handling of the particular, compositional structure of data resulting from tests built around a limited-choice and/or forced-choice procedure not only allows a formally correct analysis, but also a more efficient and coherent use of data than has hitherto been appreciated. When combined with novel and efficient testing procedures that can account for the intrinsically multivariate structure of the data (Szekely and Rizzo, 2009; Rizzo and Szekely, 2010), this adds substantially to overall sensitivity. Significantly, this gain in sensitivity results from a deep probing of information encoded in the extent and form of variance (or covariance) of choice behaviors, thus opening up an avenue for better description, and understanding, of behavioral structure.

While the examples chosen here to introduce the methodology focus on a three-choice procedure, the techniques are readily extended to situations with more than three choices (and also simplified for analysis of a dual-choice situation; see Heslop, 2009), although efficient visualization seems to be limited to a maximum of four choices (that can be displayed in a regular tedrahedron, i.e., the three-dimensional equivalent of the ternary diagram).

The realization that many behavioral tests for small animal models, including the tests for sociability and social novelty preference used as an example in the present communication, are incarnations of a much more general class of limited-choice procedures has consequences beyond statistical analysis. In fact, it allows us to view these tests in a much broader framework, and to see their direct conceptual and structural relationships with testing procedures applied in quite diverse settings of behavioral analysis. These include, for example, tests aimed at understanding consumer preferences, say the partitioning of disposable income among an exhaustive list of mutually exclusive consumer choices in commodities (say: “food,” “travel,” “electronics” and “other”) (Aitchison, 1982; Fry et al., 2000). We would also like to point out the conceptual proximity of experimental behavioral testing to document classification using dimension-reducing conditions (e.g., relying on the frequency of a few selected “key” words) to probe author characteristics (Lebanon, 2005).

We close with a cautionary note. While at times one may be tempted to forego the advantages of compositional and multivariate analysis for the convenience of more familiar univariate methods (like ANOVA, t tests), this comes at a considerable price. In doing so, one disregards, and in fact discards, valuable information. Worse, this price does not buy what it may promise. By turning a blind eye to some parts of the data, those parts chosen for analysis do not become independent of the neglected ones: they are still bounded, and they show a restricted correlation structure (i.e., they are still compositional, or closed). As one author concluded (and nicely documented for geological data), “much of the valuable information that could be gained from quantitative analysis […] is lost forever if one attempts to ‘summarize’ the results of one's painstaking labor in the form of univariate statistics” (Weltje, 2002; see also Filzmoser et al., 2009). We hope that this primer attracts the attention of behavioral scientists to the power and potential of compositional data analysis, and vice versa motivates its further development.