Introduction
Researchers in the domain of experimental psychopathology (EPP) use the experimental methodology to study psychological processes that underlie abnormal behavior. One important line of research uses experimental models aimed at mimicking abnormal processes in healthy individuals, with the aim of studying these processes in a highly controlled way. This is ideal from an experimental perspective, but has the disadvantage of a translational gap with the clinical case. The success of this translation depends on the external validity of the model. The crucial question is: ‘To what extent do the results from the model inform us about the disorder of interest?’
External validity has received a lot of attention in the pharmacological literature because of the long history of using animal subjects to model human disorders (for the screening of new drugs)Footnote 1Footnote †. Most authors identify three criteria (face, predictive and construct validity), although there is still debate about their relative importance. We discuss these criteria and add one that is particularly relevant to EPP research (diagnostic validity).
Criteria of external validity
Face validity
Face validity represents the most straightforward criterion, as it refers to the degree of phenomenological similarity between the behavior in the model and the symptoms of the disorder (Sarter & Bruno, Reference Sarter, Bruno, D'Haenen, den Boer, Westenberg and Willner2002). However, face validity is probably the weakest criterion. Similar behaviors can have different functions across species/groups whereas different behaviors may have similar functions. In addition, disorders often cannot be modeled in their entirety, and some symptoms are extremely difficult to model directly (e.g. hallucinations in schizophrenia). Face validity is neither a necessary nor a sufficient criterion of external validity.
Predictive validity
Predictive validity means that performance in the model predicts performance in the disorder (Joel, Reference Joel2006). In pharmacological research, the aim is often to predict how symptoms of a disorder will react to a novel drug compound. Predictive validity is based on testing the behavior in the model with drugs that have been shown to effectively treat the disorder. The behavior in the model should react to the drug in a similar way. This criterion is more stringent than face validity. For example, a rat model of hallucinations may lack face validity but react strongly to drugs that are known to reduce hallucinations in schizophrenia (D2 antagonists; Nestler & Hyman, Reference Nestler and Hyman2010). However, the model may also react to known treatments through a mechanism that is not related to the disorder. In such cases, the model will fail to promote further developments in the field and the validity will remain restricted to the specific drugs that originally supported the usefulness of the model (Sarter & Bruno, Reference Sarter, Bruno, D'Haenen, den Boer, Westenberg and Willner2002). Predictive validity is therefore a necessary but not a sufficient criterion.
It is important to note that applying a treatment to a model often requires adaptation of the treatment to the boundaries of the model. For example, drugs are often administered chronically to patients but acutely in the rat model (Nestler & Hyman, Reference Nestler and Hyman2010). Similarly, an EPP researcher will have to select the working ingredients from a psychotherapeutic treatment and adapt these to the model. Testing the predictive validity of a model requires a theory of the applied treatment.
Construct validity
Construct validity refers to the disease relevance of the methods by which the model is constructed, with a focus on recreating the etiological process in the model (Nestler & Hyman, Reference Nestler and Hyman2010). This requires an elaborated (etiological) theory of the disorder and of the model, and theoretical reasons to assume that the process in the model parallels the clinical process of interest. For example, prepulse inhibition tests of attention are not directly related to hallucinations (weak face validity), but they respond well to known drug treatments (D2 antagonists; strong predictive validity) and there is an elaborated theory that links attentional dysfunctions to the production of hallucinations (Lubow, Reference Lubow2005; strong construct validity).
Diagnostic validity
The use of human participants in EPP research provides the unique possibility of testing patients in the model as well. To the extent that the behaviors differ from healthy individuals (in intensity or frequency), the model has diagnostic validity and can be used as a diagnostic marker. Diagnostic validity adds to external validity because it shows that the model taps into processes that are unique to patients. However, behavioral differences are not necessarily crucial to the disorder (but, for example, reflect sociodemographic differences). Diagnostic validity supports external validity, but it is not a sufficient criterion in itself.
In conclusion, the validity of an experimental model is determined primarily by its predictive, diagnostic and construct validity. Testing the model with established treatments and with patients can evaluate the first two empirically. Construct validity has to be evaluated on theoretical grounds. Face validity is generally not important. Pharmacological models are often well tested for predictive validity, but construct validity receives much less attention (Sarter & Bruno, Reference Sarter, Bruno, D'Haenen, den Boer, Westenberg and Willner2002). By contrast, EPP models are often built on elaborated psychological theories (strong construct validity), but predictive validity has hardly been tested. In the following sections we illustrate the validation process further by applying the criteria set to two EPP models of anxiety and depression.
Application to the fear-conditioning model of anxiety
An experience of panic inside an elevator may produce fear of elevators, based on the formation of an association between elevators and panic attacks. This is modeled in fear conditioning, where a neutral stimulus (e.g. a tone) is followed by a threatening stimulus (e.g. an electrical shock). This typically produces fear of the tone (increased heartbeat, sweat gland activity, startle ability, avoidance), akin to clinical anxiety symptoms. These observations add to the face validity of the model.
The predictive validity is relatively well supported. Exposure treatments reduce clinical anxiety by exposing a patient to their phobic stimulus. Similarly, exposures to a fear-conditioned stimulus reduce fear reactions (extinction; Bouton, Reference Bouton2002). In addition, the human fear-conditioning model is sensitive to anxiolytic drugs, but the effects vary across procedures and drug compounds (Grillon, Reference Grillon2008). More systematic studies are needed.
The model has strong construct validity. The theory of the disorder and the model converge (the formation of an association between a neutral and a threatening stimulus). Furthermore, fear conditioning recruits a similar neurocircuitry as in phobic patients. Most importantly, fear-conditioned stimuli increase amygdala activity in a similar way as phobic stimuli (Sehlmeyer et al. Reference Sehlmeyer, Schöning, Zwitserlood, Pfleiderer, Kircher, Arolt and Konrad2009).
Finally, the fear-conditioning model has good diagnostic validity. A meta-analysis indicated that anxiety patients display a deviant pattern of fear reactions in the model, especially during periods of safety (Lissek et al. Reference Lissek, Powers, McClure, Phelps, Woldehawariat, Grillon and Pine2005). This suggests specific deficits in safety learning. Hence, testing diagnostic validity not only evaluates the model but also informs the theory of the disorder.
Application to the cognitive reactivity (CR) model of depression
CR refers to the degree to which an (experimentally induced) sad mood triggers negative thinking patterns as observed in clinical depression (Scher et al. Reference Scher, Ingram and Segal2005). This constitutes the face validity of the model.
The predictive validity of the CR model has not been tested systematically. Kuehner et al. (Reference Kuehner, Huffziger and Liebsch2009) found that student participants induced to ruminate (modeling depressotypic thinking) showed a significant increase in CR, whereas participants experimentally induced into a mindful self-focus (modeling an evidence-based intervention technique) showed a marginally significant decrease in CR. Further testing of depression treatments in the CR model is warranted.
The CR model has strong construct validity. First, the reinstatement of negative thinking patterns is a conceptual analog to the (cognitive) theory of depression, which precisely states that a depressive episode is triggered when a mild sad mood state reactivates negative thinking in vulnerable individuals. Second, CR is associated with biological markers of depression vulnerability (Booij & van der Does, Reference Booij and van der Does2007) and genes that are implicated in the pathophysiology of depression (Antypa & van der Does, Reference Antypa and van der Does2010). Third, Raes et al. (Reference Raes, Dewulf, Van Heeringen and Williams2009) found that an evidence-based treatment to reduce depression relapse also reduced self-reported CR in a mixed sample including recovered depressed individuals and non-clinically depressed individuals (compared to a matched control group). This supports the construct validity of the CR model by showing that CR is indeed an important aspect of depressionFootnote 2.
Finally, the CR model has strong diagnostic validity; formerly depressed patients show elevated CR levels in this model compared to never depressed controls (Miranda et al. Reference Miranda, Gross, Persons and Hahn1998). High CR is also predictive of later relapse (Segal et al. Reference Segal, Kennedy, Gemar, Hood, Pedersen and Buis2006).
Conclusions
The relevance of an EPP model depends on its external validity. We have shown that this is based primarily on two empirical criteria (predictive and diagnostic validity) and a theoretical criterion (construct validity). Face validity is generally not important. Whereas most EPP models have strong construct validity (and often diagnostic validity), tests of predictive validity are generally lacking. Systematic validation of EPP models will structure the array of models in this field, stimulate further developments in models and theories of psychopathology, and facilitate screenings of novel intervention techniques. The validation focus may catalyze the transformation of theoretical innovations into practical innovations.
Acknowledgments
Preparation of this paper was supported by a KULeuven grant (PF) – 10/005.
Declaration of Interest
None.
