Oppositional defiant disorder (ODD), defined as a pattern of angry/irritable mood, argumentative/defiant behavior, and vindictiveness, is one of the leading causes of referrals to mental health treatment for children and adolescents (Loeber et al., Reference Loeber, Burke, Lahey, Winters and Zera2000). As one of the most commonly diagnosed childhood disorders (Egger & Angold, Reference Egger and Angold2006), early manifestations of ODD comprise one developmental pathway to more serious externalizing disorders (i.e., conduct disorder [CD]), particularly for males (Burke et al., Reference Burke, Loeber and Birmaher2002; Kimonis et al., Reference Kimonis, Frick, McMahon, Mash and Barkley2014; Rowe et al., Reference Rowe, Costello, Angold, Copeland and Maughan2010). While ODD is often regarded as a “childhood-specific” disorder, symptoms of ODD can persist well into adolescence (Maughan et al., Reference Maughan, Rowe, Messer, Goodman and Meltzer2004). Adolescents with ODD tend to have poor psychiatric and psychosocial outcomes later in adolescence and into adulthood (e.g., higher levels of both internalizing and externalizing disorders, poor family attachment, criminal behavior in adulthood; Aebi et al., Reference Aebi, Plattner, Metzke, Bessler and Steinhausen2013; Copeland et al., Reference Copeland, Shanahan, Costello and Angold2009; Craig et al., Reference Craig, Hernandez, Moretti and Pepler2021; Leadbeater et al., Reference Leadbeater, Thompson and Gruppuso2012; Nock et al., Reference Nock, Kazdin, Hiripi and Kessler2007). However, adolescents are rarely the primary focus of empirical investigations of ODD, and as a result, we have a poor understanding of how ODD symptoms manifest during this critical developmental period.
Historically, some scholars have expressed concern that the diagnostic criteria for ODD overpathologize normative behavior, capturing children and adolescents with elevated, but not clinically significant, behavior problems (Kimonis et al., Reference Kimonis, Frick, McMahon, Mash and Barkley2014; Wakschlag et al., Reference Wakschlag, Briggs-Gowan, Carter, Hill, Danis, Keenan, McCarthy and Leventhal2007). These issues, coupled with evidence regarding symptom heterogeneity and dimensionality of ODD (see Frick and Nigg, Reference Frick and Nigg2012, for a review), led to revisions to the diagnostic criteria for ODD in the 5th edition of the Diagnostic and Statistical Manual of Mental Disorders (DSM 5; American Psychiatric Association, 2013). Specifically, the symptoms of ODD are now conceptualized as three separate but related dimensions – angry/irritable mood, argumentative/defiant behavior, and vindictiveness. This dimensional approach to ODD defines both the behavioral (e.g., argues with authority figures) and affective (e.g., loses temper) features of the disorder. However, release of the DSM 5 amplified debates regarding the diagnostic structure of ODD and intensified concerns of diagnostic confusion stemming from symptom overlap among ODD, bipolar disorder, and disruptive mood dysregulation disorder. Several large-scale empirical reviews of the dimensional nature of ODD have highlighted that irritability is a core feature of ODD (Evans et al., Reference Evans, Burke, Roberts, Fite, Lochman, de la Peña and Reed2017; Lochman et al., Reference Lochman, Evans, Burke, Roberts, Fite, Reed, de la Peña, Matthys, Ezpeleta, Siddiqui and Garralda2015; Mayes et al., Reference Mayes, Waxmonsky, Calhoun and Bixler2016), and as a result the International Statistical Classification of Disease and Related Health Problems (ICD-11; World Health Organization, 2019) now includes a “with chronic irritability-anger” specifier for ODD. Despite these diagnostic revisions, there remains little agreement as to whether ODD is a dimensional or categorical construct (Barry et al., Reference Barry, Marcus, Barry and Coccaro2013; Pardini et al., Reference Pardini, Frick, P. and Moffitt2010).
Dimensionality of oppositional defiant disorder
Across a variety of measures, methods, and samples, several variable-centered factor models provide robust support for the dimensionality of ODD (e.g., Aebi et al., Reference Aebi, Müller, Asherson, Banaschewski, Buitelaar, Ebstein, Eisenberg, Gill, Manor, Miranda, Oades, Roeyers, Rothenberger, Sergeant, Sonuga-Barke, Thompson, Taylor, Faraone and Steinhausen2010; Burke & Loeber, Reference Burke and Loeber2010; Burke et al., Reference Burke, Loeber, Lahey and Rathouz2005, Reference Burke, Hipwell and Loeber2010; Rowe et al., Reference Rowe, Costello, Angold, Copeland and Maughan2010; Stringaris & Goodman, Reference Stringaris and Goodman2009a, Reference Stringaris and Goodman2009b), making important distinctions between the affective (e.g., negative affect, irritability) and behavioral (e.g., oppositional, headstrong/spiteful, antagonistic, hurtful) aspects of this disorder. In fact, recent findings (Burke et al., Reference Burke, Boylan, Rowe, Duku, Stepp, Hipwell and Waldman2014; Evans et al., Reference Evans, Burke, Roberts, Fite, Lochman, de la Peña and Reed2017; Waldman et al., Reference Waldman, Rowe, Boylan and Burke2021) regarding the diagnostic structure of ODD suggest that the symptoms of ODD are heterogenous and are better accounted for by a general ODD factor (shared variance) as well as two dimensions capturing irritability and oppositional behavior (unique variance). Determining the most accurate conceptualization of ODD has important implications for the clinical assessment and treatment of ODD symptoms, as well as for understanding the developmental course of the disorder and associated psychopathologies. What remains a challenge is that studies that have applied a variable-centered approach (e.g., confirmatory factor analysis) to examine the statistical fit of various dimensional models of ODD have reached different conclusions. Specifically, some of these studies indicated similar fit across all tested modelsFootnote 1 (Evans et al., Reference Evans, Pederson, Fite, Blossom and Cooley2016; Ezpeleta et al., Reference Ezpeleta, Granero, de la Osa, Penelo and Domènech2012; Gomez & Stavropoulos, Reference Gomez and Stavropoulos2018; Ollendick et al., Reference Ollendick, Booker, Ryan and Greene2018), whereas others suggested that either a two-factor model of oppositional behavior and negative affect (Herzhoff & Tackett, Reference Herzhoff and Tackett2016; Lavigne et al., Reference Lavigne, Bryant, Hopkins and Gouze2015; Leadbeater & Homel, Reference Leadbeater and Homel2015) or a three-factor model of irritability, argumentative behavior/headstrong, and vindictiveness/hurtful (Krieger et al., Reference Krieger, Polanczyk, Goodman, Rohde, Graeff-Martins, Salum and Stringaris2013; Wesselhoeft et al., Reference Wesselhoeft, Stringaris, Sibbersen, Kristensen, Bojesen and Talati2019) provided the best fit to the data. Discrepancies in these findings limit the clinical utility of such variable-centered approaches for understanding the diagnostic structure of ODD.
Furthermore, a common criticism of these variable-centered approaches is that they capitalize on natural variability in behavior, which is then interpreted as being indicative of distinct factors. Variable-centered approaches also often rely on arbitrary cutoff values (e.g., one standard deviation above or below the mean) to determine which individuals are displaying clinically significant levels of mental health problems. This practice can result in classification errors and variability in prevalence estimates across studies. Person-centered approaches (e.g., latent class analysis [LCA]) have several strengths in comparison with more traditional variable-centered approaches, including the ability to distinguish among groups of people who differ in their probability of endorsing individual symptoms. By examining heterogeneity in behavioral patterns, person-centered approaches enable researchers to assign individuals to distinct subgroups based on their behavioral risk (for a review of these strengths, see Petersen et al., Reference Petersen, Qualter and Humphrey2019). These approaches therefore offer a useful method of understanding the underlying structure of ODD.
To our knowledge, eight published studies have utilized LCA to examine ODD symptomatology in community (Althoff et al., Reference Althoff, Kuny-Slock, Verhulst, Hudziak and van der Ende2014; Gomez & Stavropoulos, Reference Gomez and Stavropoulos2018; Herzhoff & Tackett, Reference Herzhoff and Tackett2016; Kuny et al., Reference Kuny, Althoff, Copeland, Bartels, Van Beijsterveldt, Baer and Hudziak2013; Wesselhoeft et al., Reference Wesselhoeft, Stringaris, Sibbersen, Kristensen, Bojesen and Talati2019), clinical (Burke, Reference Burke2012; Roetman et al., Reference Roetman, Siebelink, Vermeiren and Colins2021), and incarcerated (Aebi et al., Reference Aebi, Barra, Bessler, Steinhausen, Walitza and Plattner2016) samples (see Supplemental Table). While three-class solutions (i.e., low or no symptoms, irritability/combined, and oppositional) were established in two studies utilizing North American samples (Burke, Reference Burke2012; Herzhoff & Tackett, Reference Herzhoff and Tackett2016), four-class solutions (i.e., low or no symptoms, high on all symptoms, irritability, and defiant behavior) provided the best fit to the data in Dutch (Althoff et al., Reference Althoff, Kuny-Slock, Verhulst, Hudziak and van der Ende2014; Kuny et al., Reference Kuny, Althoff, Copeland, Bartels, Van Beijsterveldt, Baer and Hudziak2013), Swiss (Aebi et al., Reference Aebi, Barra, Bessler, Steinhausen, Walitza and Plattner2016) and Nordic (Wesselhoeft et al., Reference Wesselhoeft, Stringaris, Sibbersen, Kristensen, Bojesen and Talati2019) samples. In contrast, results of two studies suggested gradients in severity rather than type of ODD symptoms: a two-class solution (low and high endorsement of ODD symptoms) identified in a sample of Australian children (Gomez & Stavropoulos, Reference Gomez and Stavropoulos2018), and a three-class solution (low, medium, and high endorsement of ODD symptoms) documented in a clinical sample of Dutch children and adolescents (Roetman et al., Reference Roetman, Siebelink, Vermeiren and Colins2021).
Previous studies utilizing LCA have largely examined the structure of ODD at one point in time (cf. Kuny et al., Reference Kuny, Althoff, Copeland, Bartels, Van Beijsterveldt, Baer and Hudziak2013) during childhood (cf. Aebi et al., Reference Aebi, Barra, Bessler, Steinhausen, Walitza and Plattner2016), and therefore do not shed light on potentially important developmental shifts in the disorder (Burke & Loeber, Reference Burke and Loeber2010). In fact, evidence from latent growth curve models suggests that while defiant behavior symptoms tend to decrease over time, irritability remains relatively stable across childhood and into adolescence (Leadbeater & Homel, Reference Leadbeater and Homel2015). These findings have implications for the latent structure of ODD in adolescence, as changes in symptom endorsements may translate into different ODD classes in adolescence as compared to childhood. We cannot assume that class structures established with younger samples will hold for adolescent samples. However, the possibility that different latent classes might emerge in adolescence has not been systematically examined, as most previous studies of ODD sampled either elementary-aged children or a mix of children and adolescents. The current study aims to address this important limitation.
Differential associations between ODD and internalizing and externalizing disorders
Both the dimensions and classes of ODD identified in the extant literature demonstrate concurrent and predictive associations with other forms of child and adolescent psychopathology. Specifically, the affective dimensions/classes are more consistently associated with internalizing problems, including depression, withdrawal, anxiety, and suicidality, whereas the behavioral dimensions/classes are frequently linked to externalizing behaviors, including CD, hyperactivity, impulsivity, substance use, and criminality (Aebi et al., Reference Aebi, Plattner, Metzke, Bessler and Steinhausen2013; Althoff et al., Reference Althoff, Kuny-Slock, Verhulst, Hudziak and van der Ende2014; Burke, Reference Burke2012; Burke et al., Reference Burke, Hipwell and Loeber2010; Déry et al., Reference Déry, Lapalme, Jagiellowicz, Poirier, Temcheff and Toupin2017; Evans et al., Reference Evans, Pederson, Fite, Blossom and Cooley2016; Ezpeleta et al., Reference Ezpeleta, Granero, de la Osa, Penelo and Domènech2012; Herzhoff & Tackett, Reference Herzhoff and Tackett2016; Kuny et al., Reference Kuny, Althoff, Copeland, Bartels, Van Beijsterveldt, Baer and Hudziak2013; Lavigne et al., Reference Lavigne, Gouze, Bryant and Hopkins2014; Rowe et al., Reference Rowe, Costello, Angold, Copeland and Maughan2010; Stringaris & Goodman, Reference Stringaris and Goodman2009a, Reference Stringaris and Goodman2009b; Wesselhoeft et al., Reference Wesselhoeft, Stringaris, Sibbersen, Kristensen, Bojesen and Talati2019; Whelan et al., Reference Whelan, Stringaris, Maughan and Barker2013). However, recent research suggests that these predictive findings are not as straightforward as previously thought, as both the affective and behavioral dimensions/classes of ODD have been linked to internalizing and externalizing behaviors (Aebi et al., Reference Aebi, Barra, Bessler, Steinhausen, Walitza and Plattner2016; Drabick & Gadow, Reference Drabick and Gadow2012; Leadbeater & Homel, Reference Leadbeater and Homel2015; Mikolajewski et al., Reference Mikolajewski, Taylor and Iacono2017). These findings are consistent with suggestions that ODD reflects a common substrate underlying internalizing and externalizing disorders (Burke & Loeber, Reference Burke and Loeber2010; Burke et al., Reference Burke, Loeber, Lahey and Rathouz2005; Hipwell et al., Reference Hipwell, Stepp, Feng, Burke, Battista, Loeber and Keenan2011).
While some studies suggest that associations between ODD and later anxiety and depression are largely accounted for by initial levels of internalizing disorders (Lavigne et al., Reference Lavigne, Gouze, Bryant and Hopkins2014), others indicate that ODD adds unique prediction to internalizing disorders (Burke et al., Reference Burke, Loeber, Lahey and Rathouz2005; Copeland et al., Reference Copeland, Shanahan, Costello and Angold2009). In terms of the current DSM 5 model (Stringaris & Goodman, Reference Stringaris and Goodman2009b), only the associations between (a) the irritable dimension and later anxiety and depression and (b) the headstrong dimension and later CD remained after accounting for baseline psychopathology. Even though these studies yielded discrepant conclusions regarding the predictive associations of ODD, they all indicate that it is particularly important for analyses designed to determine the unique prospective contributions of ODD dimensions to control for initial levels of, or concurrent associations with, internalizing and externalizing disorders.
The current study
The goal of the current study is to examine the latent structure and predictive utility of ODD symptomatology in a community-based sample of adolescents. This focus on adolescence is informative given that much research has focused on the diagnostic presentation of ODD in early and middle childhood. Given that ODD in adolescence portends a range of later negative outcomes into adulthood (e.g., Craig et al., Reference Craig, Hernandez, Moretti and Pepler2021; Leadbeater et al., Reference Leadbeater, Thompson and Gruppuso2012; Nock et al., Reference Nock, Kazdin, Hiripi and Kessler2007), understanding how the symptoms of this disorder manifest in adolescence may suggest important intervention targets. If a different structure emerges for adolescents than what has been documented for younger children, then perhaps different therapeutic approaches are needed depending on developmental stage. This study therefore aims to address several of the limitations of prior ODD research by incorporating a community-based sample with sex and racial diversity in which self (i.e., adolescent)- and parent-reported internalizing and externalizing psychopathology status has been documented with well-validated measures over the course of early adolescence (grades 6–9; approximate ages 12–15 years). Given that parents and adolescents often provide markedly different accounts of adolescent behavior (De Los Reyes & Kazdin, Reference De Los Reyes and Kazdin2005), we examined both parent and self-reports of ODD separately in all models. This ability to examine both self- and parent-reported psychopathology, and to therefore consider potential informant discrepancies, is a particular strength of our study given that most previous studies of ODD have only utilized parent report of symptoms (e.g., Althoff et al., Reference Althoff, Kuny-Slock, Verhulst, Hudziak and van der Ende2014; Burke, Reference Burke2012; Kuny et al., Reference Kuny, Althoff, Copeland, Bartels, Van Beijsterveldt, Baer and Hudziak2013).
For our first aim, we used LCAs to determine if specific groups of adolescents could be identified based on their ODD symptomatology. Such analyses would suggest that there are classes of adolescents within the ODD “umbrella.” Based on previous research with North American child and adolescent samples (ages 7–14 years; Burke, Reference Burke2012; Herzhoff & Tackett, Reference Herzhoff and Tackett2016), we hypothesized a similar three-class solution in our sample of adolescents. However, given that the latent structure of ODD in adolescence has not been well-classified in the broader literature, we did not have any specific hypotheses about what types of classes would emerge from our analyses. We also sought to address several gaps in the literature, first by conducting measurement invariance tests to determine if a similar latent class structure of ODD emerged across the 4 years of adolescence examined in the current study, and next by testing for sex and racial differences in the identified classes. Although the prevalence of ODD is reported to be higher for males than females in childhood and adolescence (Boylan et al., Reference Boylan, Vaillancourt, Boyle and Szatmari2007; Demmer et al., Reference Demmer, Hooley, Sheen, McGillivray and Lum2017), there is some evidence to suggest that the latent structure of ODD does not vary depending on sex (Herzhoff & Tackett, Reference Herzhoff and Tackett2016). Continued exploration of potential sex differences in the latent structure of ODD is important, particularly given evidence of differential displays of ODD symptoms based on sex, with males engaging in more behavioral symptoms like annoying and blaming others than females (Trepat & Ezpeleta, Reference Trepat and Ezpeleta2011). In terms of racial differences, Merikangas et al. (Reference Merikangas, He, Burstein, Swanson, Avenevoli, Cui, Benjet, Georgiades and Swendsen2010) conducted a large epidemiological study utilizing structured diagnostic interviews with adolescents and determined the prevalence of ODD diagnoses to be similar across racial groups. Sex and racial differences in endorsement of ODD symptoms have not been fully examined within a latent class framework; therefore, it is unknown if the latent classes postulated in the broader ODD literature apply to both males and females and across different racial groups.
The second aim of the current study was to determine if the identified classes differentially predicted later internalizing and externalizing psychopathology, controlling for baseline psychopathology levels. Given that we do not know a priori what classes will emerge from our LCA models, as well as recent research indicating that both affective and behavioral aspects of ODD predict both internalizing and externalizing behaviors, we do not present any directional hypotheses for this aim. Findings regarding any differential predictive associations may help to inform models of comorbidity and suggest intervention targets to prevent later psychopathology.
Method
Participants
Data came from the Developmental Pathways Project, a community-based prospective cohort study examining depression and co-occurring conduct problems in a diverse sample of adolescents. In the first phase of the study, four middle schools were selected whose students reflected the sociodemographic backgrounds of those enrolled in the Seattle Public School District. Students in these schools completed a mental health screening questionnaire to assess for symptoms of depression and conduct problems (for details regarding these screening procedures, see Vander Stoep et al., Reference Vander Stoep, McCauley, Thompson, Herting, Kuo, Stewart, Anderson and Kushner2005). Universal screening was conducted in these schools for four successive years (2001–2005), during which 2190 6th grade students were screened. Adolescents with limited English proficiency, severe developmental disabilities, or whose parents declined to give permission for their child to participate were excluded from the screening sample.
Students were then stratified into four subgroups based on their screening scores: (1) comorbid group, with elevated scores on both depression and conduct problem measures; (2) depressed group, with elevated depression scores only; (3) conduct problems group, with elevated conduct problem scores only; and (4) low group, with low scores on both depression and conduct problem measures. The cutoff defining elevation was 0.5 SD above the screening sample mean on the Mood and Feelings Questionnaire (Costello & Angold, Reference Costello and Angold1988) for depression (score of 15; 24.5% of the sample) and/or the externalizing subscale of the Youth Self-Report (Achenbach & Rescorla, Reference Achenbach and Rescorla2001) for conduct problems (raw score of 12; 20.8% of the sample). A random sample of adolescents was then selected from these groups in a ratio of 1:1:1:2, respectively, to participate in the longitudinal phase of the study. This sampling strategy resulted in an overrepresentation of youth in the three psychopathology groups compared to what would be expected in the general population. We then used two-component sampling weights to account for this oversampling of youth with elevated psychopathology scores. Each adolescent in the longitudinal cohort sample received two weights – (1) a sampling fraction weight equivalent to the inverse probability of being enrolled in the study based on the four psychopathology screening groups; and (2) a post-stratification weight that accounted for differences in sex, race, and educational program between the screening and longitudinal samples. These two values were multiplied together to produce a final weight for each adolescent in the current sample. The application of this weight ensured that our longitudinal sample was comparable to the full screening sample and was representative of middle school students in the Seattle Public School District.Footnote 2 These recruitment, sampling, and weighting procedures mirror those used in other longitudinal child psychiatric epidemiological research studies (e.g., Fast Track Project [Conduct Problems Prevention Research Group, 2020]; Great Smoky Mountains Study [Costello et al., Reference Costello, Angold and Burns1996]). In all of these studies, a weighting variable was created and used in analyses to ensure that the sample was representative of the community from which the participants were recruited.
Of the 913 adolescents recruited for the longitudinal cohort study, 807 were eligible for participation (exclusion criteria included limited parental English proficiency or family moving out of the study area), and 521 adolescents and a parent/guardian agreed to participate. Of eligible adolescents, those who participated in the cohort study were more likely to be White, to be in a gifted educational program, and to have a parent born in the United States. However, there were no differences in terms of sex, Hispanic ethnicity, and mean scores on the screening measures.
Sample
The current analyses utilized data from the cohort study (n = 521 adolescents). Adolescents were on average 12.01 (SD = 0.43) years of age at the baseline interview (i.e., 6th grade). The sample reflected the racial, ethnic, and economic characteristics of the Seattle Public School District. Specifically, approximately half of the sample was male (51.9%) and White (48.7%). Within the sample, 28.2 % were Black, 18.5% Asian, and 4.0% Native American; 10.4% identified as Latinx. Of the participating parents/guardians, 394 (78.6%) were biological mothers, 70 (15.2%) were biological fathers, and the remaining were adoptive or foster parents (3.3%) or other relatives/adults (6.0%). Approximately half of the parents/guardians reported obtaining a Bachelor’s degree or higher (n = 251, 48.2%), with 148 completing some college (28.5%), and the remainder obtaining a high school degree/GED or less (n = 121, 23.2%). In terms of household income, 139 (26.7%) reported an income of $25,000 or less, 121 (23.2%) reported an income between $25,000 and $49,000, 99 reported an income between $50,000 and $74,999 (19.0%), and the remainder reported an income greater than $75,000 (n = 162, 31.1%).
Procedure
Written consent and assent were obtained from parents/guardians and adolescents, respectively, for their participation in the study. Adolescents and their participating parent/guardian were interviewed separately by two research assistants in private locations in the family’s place of residence or other location convenient for the family. Baseline interviews (i.e., 6th grade) were conducted approximately 3 months after the initial universal mental health screening, and additional interviews were conducted every 6 months thereafter until the adolescents were in the 8th grade. Then an interview was conducted 12 months later in 9th grade, for a total of six interviews. Interviewers received extensive training in the administration of study measures and completed a certification process whereby they participated in mock interviews and conducted their first field interview with an experienced interviewer. Reliability and quality assurance checks were conducted, and each interviewer participated in weekly supervision meetings. The Institutional Review Board of the University of Washington approved all study procedures.
Measures
Demographics
Demographic characteristics that were included as covariates in the current study were sex (1 = male, 2 = female) and race (1 = Black, 2 = White, 3 = Other). Since the majority of participants were either Black or White, two sets of contrast codes were created for analyses including race: (1) Black compared to White adolescents, and (2) Black and White adolescents compared to adolescents in the “Other” (i.e., Native American, Asian, Pacific Islander) race categories. Latinx youth, who comprised 10% of the study sample, were classified according to their racial group.
Diagnostic interview schedule for children (DISC-IV)
The DISC-IV (Shaffer et al., Reference Shaffer, Fisher, Lucas, Dulcan and Schwab-Stone2000) is a 3000-item, standardized structured diagnostic interview designed to assess the prevalence of 34 anxiety, mood, behavioral, and substance use disorders in children and adolescents according to DSM-IV criteria. The DISC-IV interview obtains information regarding the presence of symptoms (yes/no) within the past year and past month, as well as their frequency (ranging from “never” to “almost every day”), duration, and severity. Given that multiple interview response items were used to determine if a particular DSM-IV symptom has been endorsed, both symptom and frequency/duration counts were recorded. The reliability and validity of the DISC-IV has been well-established (Shaffer et al., Reference Shaffer, Fisher, Dulcan, Davies, Piacentini, Schwab-Stone, Lahey, Bourdon, Jensen, Bird, Canino and Regier1996, Reference Shaffer, Fisher, Lucas, Dulcan and Schwab-Stone2000). Given the 12-month time frame of the DISC-IV interview, the ODD, CD, and depression modules were administered to both adolescent and parent participants every 12 months, at the 1st, 3rd, 5th and 6th interviews, which were conducted in 6th, 7th, 8th, and 9th grades. Given the focus of the current study, we specifically examined the presence of past-year symptoms (yes/no) from the ODD, CD, and depression modules.
The DISC-IV ODD module asked questions about 12 specific behaviors related to ODD, which we then combined to generate data regarding the 8 symptoms of ODD as defined in the DSM-IV (i.e., loses temper, argues with authority figures, actively defies or refuses to comply, deliberately annoys others, blames others for mistakes, touchy/easily annoyed, angry/resentful, spiteful and vindictive).Footnote 3 Specifically, the DISC-IV scoring algorithm specifies the following three stem-level item combinations: (1) the items “done things on purpose that caretakers said not to do” and “refused to do things that caretakers said to do” are combined to create the “actively defies or refuses to comply” ODD symptom; (2) the items “seemed mad at people or things” and “felt things were unfair and got angry” are combined to create the “angry/resentful” ODD symptom; and (3) the items “done mean things to people on purpose” and “gotten even with people by messing up their things/hurting them” are combined to create the “spiteful and vindictive” ODD symptom. These item-level combinations were constructed using the “or” rule (i.e., we counted the symptom as being present if either of the two original stem-level items were endorsed) (Piacentini et al., Reference Piacentini, Cohen and Cohen1992). To adhere to the DSM-IV diagnostic criteria for ODD and to ensure orthogonality of items for the LCAs, we used the 8 DISC-IV generated ODD symptom items (as opposed to the original 12 interview response items) in all analyses. To be consistent with other LCA studies that utilized data from the DISC-IV (Burke, Reference Burke2012; Herzhoff & Tackett, Reference Herzhoff and Tackett2016), we did not factor the “frequency” or “duration” responses into the ODD symptom variables. We utilized a similar procedure to distill the interview response items from the DISC-IV depression and CD modules (21 and 40 items, respectively) down to the DSM-IV diagnostic symptoms of major depressive disorder and CD (9 and 15 symptoms, respectively).Footnote 4
Overview of analyses
Descriptive analyses were conducted in SPSS version 28, and the latent variable analyses were performed in Mplus 8.4 (Muthén & Muthén, Reference Muthén and Muthén2019). All analyses incorporated the weighting variable described above (a rescaling factor was used for analyses in SPSS).
Missing data and attrition
The flowchart in Figure 1 presents the sample sizes at each time point analyzed in the current study. As with many longitudinal studies, there is a degree of missing data in the current study, due largely to non-participation/attrition at follow-up assessments (i.e., 7th, 8th, and 9th grades). Missing data were handled with full-information maximum likelihood in Mplus, which is robust to the presence of missing data when they are missing at random (MAR; Little & Rubin, Reference Little and Rubin2002). We also used a maximum likelihood estimator that calculated robust standard errors (MLR).
Figure 1. Flow chart indicating sample sizes at each time point analyzed in the current study.
In terms of demographics, adolescents with missing adolescent and parent data at 7th grade were more likely to be Black as compared to White, χ2 adolescent missing (1, n = 403) = 18.47, p < .001, χ2 parent missing (1, n = 403) = 24.06, p < .001, and to be in the Other race categories as compared to White, χ2 adolescent missing (1, n = 373) = 12.11, p < .01, χ2 parent missing (1, n = 373) = 15.79, p < .001. Adolescents with missing adolescent and parent data at 8th grade were more likely to be Black as compared to White, χ2 adolescent missing (1, n = 403) = 13.98, p < .001, χ2 parent missing (1, n = 403) = 17.40, p < .001. Those with missing parent data at 8th grade were also more likely to be in the Other race categories as compared to White, χ2 (1, n = 373) = 5.59, p < .05. Adolescents with missing parent data at 9th grade were more likely to be Black as compared to White, χ2 (1, n = 403) = 9.98, p < .01, and to be in the Other race categories as compared to White, χ2 (1, n = 373) = 9.37, p < .01.
In terms of baseline (i.e., 6th grade) DISC-IV symptoms, adolescents with missing adolescent and parent data at 7th grade had fewer parent-reported symptoms of ODD, t(93.47)adolescent missing = 3.35, p < .05, t(92.96)parent missing = 3.22, p < .01, but more self-reported symptoms of CD, t(91.11)adolescent missing = −2.92, p < .01, t(88.94)parent missing = −3.19, p < .01. Adolescents with missing adolescent and parent data at 8th grade had more self-reported symptoms of ODD, t(507)adolescent missing = −3.26, p < .01, t(507)parent missing = −3.03, p < .01, fewer parent-reported symptoms of ODD, t(89.56)adolescent missing = 2.18, p < .05, t(88.27)parent missing = 2.35, p < .05, and more self-reported symptoms of CD, t(89.11)adolescent missing = −3.51, p < .01, t(89.01)parent missing = −3.39, p < .01. Adolescents with missing adolescent and parent data at 9th grade had more self-reported symptoms of CD, t(161.75)adolescent missing = −1.99, p < .05, t(151.26)parent missing = −2.96, p < .01.
Latent class analyses (LCAs)
LCA (McCutcheon, Reference McCutcheon1987) is a person-centered, probabilistic method for identifying patterns of behavioral heterogeneity. The method uses estimated posterior probabilities to classify groups of individuals into an optimal number of latent classes or subgroups. The number of classes is determined by various factors (e.g., parsimony, sample size) and requires consideration of theory, prior research, and fit statistics (Weller et al., Reference Weller, Bowen and Faubert2020). In the current study, classes of the adolescents’ oppositional behavior were based on the eight symptoms of ODD described earlier. We conducted separate LCAs across all four grades (i.e., 6th through 9th grade) according to both adolescent and parent report, for a total of eight LCA models. For each LCA, class solutions were tested iteratively (i.e., 1, 2, 3, and 4 latent classes) to determine the best-fitting model. Model estimation ceased when fit indices did not improve or began to level off, and/or when the extra class was not qualitatively different from the other classes. To determine model fit, we examined the Bayesian Information Criteria (BIC), sample size adjusted BIC (SSA BIC), and Lo-Mendell Rubin likelihood ratio test (LMR-LRT). For the BIC and SSA BIC, lower values indicate a better fit to the data. The LMR-LRT compares the relative fit of a model with k classes to a model with k-1 classes. A large probability (i.e., p > .05) indicates that the additional profile does not significantly improve the fit of the model over the previous model with one less profile (Nylund et al., Reference Nylund, Asparouhov and Muthén2007); that is, a nonsignificant LMR-LRT p-value for a k class solution provides support for the k-1 class solution (Nylund-Gibson & Choi, Reference Nylund-Gibson and Choi2018). For models with similar fit indices, the most parsimonious model (i.e., with the fewest parameters) was preferred. We also examined entropy as a measure of classification accuracy; entropy values range from 0 to 1, and values closer to 1 indicate greater classification accuracy.
Measurement invariance
Measurement invariance tests the equality of the parameters in a measurement model across groups or time and is a critical step in all longitudinal analyses (Putnick & Bornstein, Reference Putnick and Bornstein2016). For our latent class models, that meant testing for equality in the conditional item probabilities to determine if the same number and types of classes occurred at all time points. For the current analyses, after identifying the best-fitting class solution we specifically tested for any differences in the item thresholds from one time point to the next (i.e., from 6th to 7th grade, from 7th to 8th grade, and from 8th grade to 9th grade). Following procedures outlined by Nylund (Reference Nylund2007), we specified three measurement models: (a) full measurement noninvariance (all item thresholds across the identified classes were freely estimated across the two time points; i.e., unconstrained, fully free), (b) full measurement invariance (item thresholds across the identified classes were constrained to be the same across both time points; i.e., fully constrained), and (c) partial measurement invariance (some item thresholds across the identified classes were constrained to be equal, and others were left unconstrained across the two time points). Using the −2-log-likelihood test (Byrne et al., Reference Byrne, Shavelson and Muthén1989; Nylund, Reference Nylund2007), we first compared a model assuming full invariance to models assuming full noninvariance and partial invariance and then compared a model assuming partial invariance to the noninvariance model, incorporating the scaling correction factor obtained with the MLR estimator (Satorra & Bentler, Reference Satorra and Bentler2010). Nonsignificant −2 log likelihood tests indicated that the more restrictive model (i.e., the model with more parameter equality constraints/more invariance) provided a better fit to the data than the less restrictive model (i.e., the model with fewer parameter constraints/less invariance).
Covariate effects and predictive associations
We next included sex and race as covariates in the LCAs to examine between-class differences. Specifically, we utilized the automatic 3-step method (i.e., R3STEP command) for latent class predictors (i.e., covariates) outlined in Asparouhov and Muthén (Reference Asparouhov and Muthén2014). This 3-step process is important given that class memberships tend to change when additional variables are introduced to the latent class measurement model, creating error and bias in parameter estimates (Nylund-Gibson et al., Reference Nylund-Gibson, Grimm, Quirk and Furlong2014, Reference Nylund-Gibson, Grimm and Masyn2019; Weller et al., Reference Weller, Bowen and Faubert2020). As such, covariates were introduced as auxiliary variables to prevent changes in class estimation. Given that the latent classes are the dependent variables and the covariates are the predictors/independent variables, the resulting coefficients are multinomial logistic regression coefficients (logits and odds ratios). These analyses explore whether class prevalence is equal across levels of these predictors of class membership (Nylund-Gibson & Choi, Reference Nylund-Gibson and Choi2018). Lastly, we used the classes identified from the self- and parent-reported LCAs to predict later self- and parent-reported symptoms of depression and CD across time. To incorporate both the weighting variable and the covariates in these distal outcome analyses, we utilized the manual 3-step LCA method outlined in Asparouhov and Muthén (Reference Asparouhov and Muthén2020). This procedure estimates a joint model incorporating both the measurement and auxiliary (including the covariates and distal outcomes) models. Following this 3-step process, pairwise mean differences in later (i.e., grades 7–9) symptoms of depression and CD were tested for significance across the latent ODD classes at previous time points (e.g., mean symptoms of depression and CD at grade 9 were tested for differences across the latent ODD classes identified at grades 6, 7, and 8). Significant omnibus Wald chi-square test statistics indicated that at least one of the pairwise mean differences in the outcomes was significantly different among the classes; follow-up analyses with individual z-tests then determined which pairwise mean differences were significant between two classes (e.g., mean symptoms of depression at grade 9 differed between two classes at grade 6). All analyses tested within- (e.g., self-reported ODD classes predicting self-reported distal outcomes) and between- (e.g., self-reported ODD classes predicting parent-reported distal outcomes) reporter models.
Results
Descriptive statistics
Bivariate correlations generally revealed positive associations between symptoms of ODD and symptoms of both depression and CD, as well as positive associations between symptoms of depression and CD (see Table 1). Higher and more consistent correlations were observed within reporter (e.g., self-reported ODD and self-reported depression were associated at all time points) as compared to cross-reporter measures (e.g., self-reported symptoms of depression at grade 8 were unrelated to parent-reported symptoms of ODD at grade 8).
Table 1. Intercorrelations among symptoms of oppositional defiant disorder, depression, and conduct disorder
Note. ODD = oppositional defiant disorder, Dep = depression, CD = conduct disorder, 6 = 6th grade, 7 = 7th grade, 8 = 8th grade, 9 = 9th grade, a = adolescent report, p = parent report.
* p < .05.
** p < .01.
*** p < .001.
Across the four grades examined in the current study, weighted past-year prevalence of ODD (as defined by the DISC-IV scoring algorithm to determine whether the adolescent met criteria for a diagnosis of ODD) ranged from 1.6% to 2.1% according to self report and from 5.9% to 9.5% according to parent report. Combined prevalence of ODD (i.e., using the “or” rule) was 12.0% at grade 6, 8.3% at grade 7, 8.2% at grade 8, and 7.3% at grade 9. These percentages are consistent with estimates of ODD diagnostic prevalence in youth as reported in large-scale epidemiological studies (e.g., 2–15% for past-year ODD prevalence per Nock et al., Reference Nock, Kazdin, Hiripi and Kessler2007; 12.6% for lifetime ODD prevalence per Merikangas et al., Reference Merikangas, He, Burstein, Swanson, Avenevoli, Cui, Benjet, Georgiades and Swendsen2010), community studies (e.g., 2.6–15.6% per Boylan et al., Reference Boylan, Vaillancourt, Boyle and Szatmari2007), and a North American community-based sample that utilized the DISC-IV to examine latent classes of ODD (i.e., 11.2% per Herzhoff & Tackett, Reference Herzhoff and Tackett2016). For depression, weighted past-year self-reported prevalence ranged from 0.2% to 1.4%, while parent-reported past-year prevalence ranged from 1.1% to 1.9% (combined ranged from 1.3% to 2.9%). Weighted past-year prevalence of CD ranged from 2.7% to 3.8% according to adolescents and from 1.1% to 2.2% according to parents (combined ranged from 4.3% to 5.4%).
Latent classes of adolescent oppositional behavior
The LCA fit statistics at all four time points from both adolescent and parent reporters generally indicated that a three-class solution provided the most parsimonious fit to the data (see Table 2). In keeping with current reporting guidelines (van de Schoot et al., Reference van de Schoot, Sijbrandij, Winter, Depaoli and Vermunt2017), these decisions were largely supported by increases in BIC levels and nonsignificant LMR-LRT p-values for the four-class solutions. However, the LMR-LRT p-value did not reach statistical significance at the three-class solution for self-reported ODD at grade 8 and parent-reported ODD at grade 9, suggesting that in these instances a two-class solution provided the best fit. However, the BIC values did not support the two-class solutions, as those values continued to decrease for the three-class solutions. When the BIC and LMR-LRT fit statistics disagree, the BIC is generally favored, given simulation studies indicating that it is more accurate for detecting the correct number of classes (Nylund et al., Reference Nylund, Asparouhov and Muthén2007). We therefore concluded that a three-class solution provided the best fit to the data across time points and reporters. Entropy and smallest class sizes (i.e., the proportion of adolescents within the smallest class) also suggested that individuals were well classified in the three-class solutions.
Table 2. Fit indices for unconditional latent class models of ODD at grades 6 through 9 according to adolescent and parent report
Note. Results from unconditional LCA models (i.e., no covariates included) are presented. ODD = oppositional defiant disorder; LCA = latent class analysis; BIC = Bayesian Information Criterion; SSA BIC = sample size adjusted BIC; LMR-LRT = Lo-Mendell Rubin adjusted likelihood ratio test p-value. Bold indicates best-fitting model.
We examined symptom endorsement patterns to aid in substantive interpretation of the classes, and a similar structure emerged at all time points according to both adolescent and parent reports (see Figure 2). The first class had the highest endorsement of all of the DISC-IV ODD symptoms and was therefore considered the “high” class. The second class followed a similar pattern of item endorsement as the high class, but the probabilities of these item endorsements were intermediate between the probabilities for the two other classes. We therefore labeled this class the “medium” class. The third class had the lowest endorsement of all DISC-IV ODD symptoms and was designated the “low” class.
Figure 2. Graphs of unconditional latent classes of symptoms of oppositional defiant disorder according to adolescent (panel a) and parent (panel b) report across grades 6–9.
Figure 2 presents the percentages of adolescents classified into each class according to both adolescent and parent reports. These class proportions are based on the posterior probabilities identified in the LCA solution. As such, individuals are assigned to classes based on their probability of being in that class given their pattern of scores on the indicator variables. The largest percentages of adolescents fell into the medium class at all four time points and across both reporters except for parent report at grade 9 (percentages ranged from 33.28% to 55.71%). The percentage of adolescents classified into the high class was consistently larger according to parent report (percentages ranged from 33.33% to 35.42%) compared to self report (percentages ranged from 19.08% to 29.38%).
Measurement invariance
The findings described above indicating that the number and types of ODD classes were the same at all four time points provided support for configural invariance across time. We next tested for non-, partial, and full measurement invariance between contiguous sets of classes (i.e., grade 6 to grade 7, grade 7 to grade 8, grade 8 to grade 9). There are many different partial measurement tests that can be performed with three latent classes at four time points. Testing all possible partial measurement specifications can lead to over-testing of the data, and so decisions about which tests to apply must consider the aims of the analysis (Nylund, Reference Nylund2007). Given our interest in understanding elevations in ODD symptomatology in our sample, we set equality constraints on the item thresholds of the high and medium classes and left the item thresholds for the low class unconstrained (essentially setting the low class as the reference class).
Examination of the results presented in Table 3 indicates support for the assumption of partial measurement invariance between grades 6 and 7 according to adolescents and full measurement noninvariance according to parents. According to both adolescents and parents, the findings support full measurement invariance between grades 7 and 8 and full measurement noninvariance between grades 8 and 9.
Table 3. Results from measurement invariance tests across subsequent grades according to adolescent and parent report
Note. −2 log likelihood tests incorporate MLR scaling correction factor (not displayed here; calculations available upon request from the first author). Partial invariance tests hold the high and medium classes equal. “Best fit” column indicates which model provides the better fit for that particular model test.
* p < .05.
** p < .01.
*** p < .001.
Sex and race differences in classes of adolescent ODD
Sex differences
Using the low class as the reference class, females were less likely to be in the high class at grade 6 according to adolescents and parents (Odds Ratio (OR adolescent ) = 0.33, 95% CI = [0.15, 0.71]; OR parent = 0.50, CI = [0.25, 0.98]) and in the medium class at grade 7 according to parent report (OR = 0.26, CI [0.11, 0.61]).
Race differences
Using the low class as the reference class, according to self report, Black adolescents, as compared to White adolescents, were more likely to be in the high (OR = 2.17, CI [1.23, 3.83]) and medium (OR = 1.82, CI [1.04, 3.20]) classes at grade 6. In contrast, by parent report, Black adolescents, as compared to White adolescents, were less likely to be in the high and medium classes at grade 6 (OR high = 0.45, CI [0.29, 0.69]; OR medium = 0.58, CI [0.39, 0.87]), grade 7 (OR high = 0.38, CI [0.22, 0.64]; OR medium = 0.56, CI [0.34, 0.92]), and grade 9 (OR high = 0.64, CI [0.44, 0.93]; OR medium = 0.61, CI [0.39, 0.95]). Black and White adolescents, as compared to the “Other” race category, were more likely to be in the high class at grade 6 (OR = 2.39, CI [1.42, 4.02]), grade 7 (OR = 2.42, CI [1.36, 4.30]), grade 8 (OR = 2.24, CI [1.15, 4.39]), and grade 9 (OR = 2.01, CI [1.12, 3.61]) according to parent report and to be in the medium class according to adolescents and parents at grade 6 (OR adolescent = 2.29, CI [1.27, 4.41]; OR parent = 2.05, CI [1.30, 3.22]) and grade 7 (OR adolescent = 1.94, CI [1.10, 3.42]; OR parent = 3.05, CI [1.67, 5.58]).
Prediction to later symptoms of depression and CD
Given the significant sex and race differences identified above, we controlled for sex, race, and baseline (i.e., 6th grade) symptoms of depression and CD in all analyses predicting later symptom counts of depression and CD from the latent classes of ODD. Before examining these predictive associations, we examined concurrent associations between the identified ODD classes (low class is the reference class) at grade 6 with baseline (i.e., grade 6) symptom counts of depression and CD. Below we highlight the main findings from these analyses.
Depression
For the concurrent associations, adolescents with higher self-reported baseline depression were more likely to be in the self-reported high and medium ODD classes at grade 6 (OR high = 2.39, CI [1.74, 3.28]; OR medium = 2.36, CI [1.80, 3.09]). Similarly, adolescents with higher parent-reported baseline depression were more likely to be in the parent-reported high and medium classes at grade 6 (OR high = 2.39, CI [1.74, 3.28]; OR medium = 1.54, CI [1.18, 2.01]). Since these are continuous predictor variables, the odds ratios reflect the incremental change in the odds of the outcome when there is a one-unit increase in baseline depression symptom count (range: 0–9). For example, a one-unit increase in self-reported baseline depression was associated with a 2.39 times higher odds of being in the self-reported medium class as compared to the low class.
Table 4 presents the results from our distal outcome analyses using the identified latent ODD classes at previous time points (i.e., grades 6 through 8) to predict later (i.e., grades 7 through 9) symptoms of depression. Predicted means are presented in the cells of this table, with subscripts indicating which pairwise means were significantly different from each other between two compared ODD classes. Consistent across both within- (e.g., self-reported latent ODD classes predicting later self-reported depression) and between- (e.g., self-reported latent ODD classes predicting later parent-reported depression) reporter models, membership in both the high and medium classes, as compared to the low class, generally predicted higher mean symptom counts of depression. At later time points (e.g., prediction to symptoms of depression in grades 8 and 9), there was also evidence that membership in the high class as compared to the medium class predicted more symptoms of depression.
Table 4. Predicting later symptoms of depression according to adolescent and parent report from the latent classes of ODD
Note. ODD = oppositional defiant disorder. Coefficients within the table are mean symptom counts of depression (standard deviations in parentheses) as predicted from the latent ODD classes at previous time points, controlling for sex, race, and baseline (i.e., 6th grade) symptoms of depression (grand mean centered). Within each grade, subscripts indicate results from follow-up individual z-tests determining which pairwise means were significantly different at p < .05 (i.e., a subscript of H indicates a significant difference from the mean in the high class, a M indicates a significant difference from the mean in the medium class, and a L indicates a significant difference from the mean in the low class).
CD
For the concurrent associations, adolescents with higher self-reported baseline CD were more likely to be in the self-reported high ODD class at grade 6 (OR = 4.40, CI [1.77, 10.93]), and adolescents with higher parent-reported baseline CD were more likely to be in the parent-reported high and medium ODD classes (OR high = 6.46, CI [3.22, 12.94]; OR medium = 4.23, CI [2.13, 8.38]). As previously stated above, these odds ratios reflect the incremental change in the odds of the outcome when there is a one-unit increase in baseline CD symptom count (range: 0–15).
As above, Table 5 presents the results from our distal outcome analyses using the identified latent ODD classes at previous time points to predict later symptoms of CD. More consistent findings were indicated for the within-reporter models, with membership in both the high and medium classes as compared to the low class, and membership in the high class as compared to the medium class, predicting higher mean symptom counts of CD. Of the few cross-reporter effects that emerged, membership in the high and medium classes predicted more symptoms of CD at later time points (e.g., prediction to CD in grades 8 and 9).
Table 5. Predicting later symptoms of conduct disorder according to adolescent and parent report from the latent classes of ODD
Note. ODD = oppositional defiant disorder. Coefficients within the table are mean symptom counts of CD (standard deviations in parentheses) as predicted from the latent ODD classes at previous time points, controlling for sex, race, and baseline (i.e., 6th grade) symptoms of CD (grand mean centered). Within each grade, subscripts indicate results from follow-up individual z-tests determining which pairwise means were significantly different at p < .05 (i.e., a subscript of H indicates a significant difference from the mean in the high class, a M indicates a significant difference from the mean in the medium class, and a L indicates a significant difference from the mean in the low class).
* No estimates due to model nonidentification (very low base rate of CD endorsement among adolescents in the Low class).
Discussion
The goal of the current study was to use person-centered analyses to investigate the latent structure and predictive utility of ODD in a community-based sample with a high representation of minority adolescents. This statistical approach allowed us to consider heterogeneity in ODD symptomatology and determine if there were different classes of ODD based on patterns of symptom endorsement. Importantly, we sought to address gaps in the knowledge base by examining the latent structure of ODD over the course of early adolescence (i.e., grades 6 through 9) and by considering potential age, sex, and race differences in the identified classes of ODD (review of the Supplemental Table highlights many of these gaps). We tested for differential prediction to later symptoms of depression and CD among the ODD classes, controlling for baseline symptom counts of those disorders. Additionally, we conducted all analyses using both self- and parent-reported symptoms to allow for comparisons of within- and cross-reporter findings.
Latent classes of ODD
We identified three classes of self- and parent-reported ODD symptomatology – high, medium, and low endorsement of ODD symptoms. While several previous studies (e.g., Althoff et al., Reference Althoff, Kuny-Slock, Verhulst, Hudziak and van der Ende2014; Kuny et al., Reference Kuny, Althoff, Copeland, Bartels, Van Beijsterveldt, Baer and Hudziak2013; Wesselhoeft et al., Reference Wesselhoeft, Stringaris, Sibbersen, Kristensen, Bojesen and Talati2019) have identified four ODD classes, these solutions were largely identified in European samples using behavioral checklists. Given that other LCA studies with North American samples have also identified three-class solutions, this discrepancy in findings suggests that there may be important differences in the structure of ODD based on the sample and measurement approach. More cross-cultural and psychometric research examining child and adolescent ODD is clearly needed, and meta-analytic approaches may help clarify the latent structure of this disorder across different samples and methodologies.
We also did not find evidence of purely affective (“irritable”) or behavioral (“oppositional”) classes. Rather, our findings highlight comorbidity in affective and behavioral symptoms of ODD and may suggest potential developmental differences in the expression of ODD. Specifically, this study is unique in its focus on adolescence, a developmental period that has been largely excluded from previous studies of ODD. Our latent class findings suggest that adolescents with high levels of behavioral risk are also more likely to exhibit irritability (i.e., touchy, angry) and argumentativeness, and that the latent structure underlying heterogeneity in these symptoms may reflect gradients of severity in ODD. These results are consistent with conceptualizations that, for some adolescents, this period of development is characterized by mood disruptions, engagement in risk behavior, and conflict with parents (Arnett, Reference Arnett1999).
Examination of the item endorsement patterns within the high and medium classes (see Figure 2) also indicate that, across time and according to both adolescents and parents, the items “temper,” “argues,” “touchy,” and “angry” exhibited a much higher probability of endorsement than the other items within those classes. This endorsement pattern may reflect that irritability and argumentativeness are often seen at higher rates in adolescence than in childhood or adulthood (Craig et al., Reference Craig, Hernandez, Moretti and Pepler2021). Additionally, it is interesting to note that the largest proportions of our study sample were consistently (with the exception of parent-report at grade 9) classified into the medium class, possibly reflecting features of emotional and behavioral dysregulation that are more commonly observed during adolescence than at other developmental periods (Silk et al., Reference Silk, Steinberg and Morris2003). Furthermore, and consistent with the higher prevalence of parent-reported ODD diagnosis estimates noted earlier, the proportions of the study sample classified into the high classes at all four time points were generally higher according to parent report as compared to self-report. This pattern of findings may reflect discrepancies in adolescent and parent reports of adolescent behavior (De Los Reyes & Kazdin, Reference De Los Reyes and Kazdin2005), or may be due to evidence of differential attrition in the current study. That is, parents who reported fewer adolescent ODD symptoms at baseline tended to attrit from the study at the follow-up assessments; thus, parents who contributed data at later time points may have been more likely to report higher levels of ODD symptoms during those assessments. Other studies have also documented that parents are more likely to drop-out of longitudinal studies when youth display lower levels of psychopathology (Eisner et al., Reference Eisner, Murray, Eisner and Ribeaud2019).
In general, it is not uncommon for LCA modeling approaches to find that a three-class solution provides the best fit to the data, which characteristically take on a “salsa” (i.e., high-medium-low) pattern. Critiques of this approach, and of three-class LCA solutions in particular, argue that this pattern of findings simply creates artificial “cut points” along an underlying continuous dimension (e.g., symptom severity), and therefore does not reflect actual heterogenous classes of individuals (Hallquist & Wright, Reference Hallquist and Wright2014). In fact, this “salsa” pattern is relatively common in psychiatric data, as symptom endorsements from clinical interviews and related diagnostic measures tend to covary, leading to solutions where the latent classes differ quantitatively but not qualitatively (Ellickson-Larew et al., Reference Ellickson-Larew, Escarfulleri and Wolf2020). While our current findings stand in contrast to others in the extant literature, some recent work has begun to conceptualize the latent classes of ODD as reflecting gradients of severity in symptom presentation (Gomez & Stavropoulos, Reference Gomez and Stavropoulos2018; Roetman et al., Reference Roetman, Siebelink, Vermeiren and Colins2021).
It is notable that our results are most consistent with those published by Roetman et al. (Reference Roetman, Siebelink, Vermeiren and Colins2021), who utilized a computerized diagnostic interview to measure ODD symptoms with a sample of clinic-referred youth aged 5–18; these similarities in findings may reflect commonalities in measurement approaches. Our findings are also consistent with IRT modeling work (Lindhiem et al., Reference Lindhiem, Bennett, Hipwell and Pardini2015) suggesting little incremental utility in using ODD symptom profiles versus symptom counts. Finally, it is important to highlight here that any latent class solution is not meant to provide a definitive set of classes, but rather to suggest potential behavioral patterns that can enhance understanding of how to conceptualize a particular construct (Lanza & Collins, Reference Lanza and Collins2006). There also tends to be a positive association between sample size and the number of classes identified in a LCA solution (Masyn, Reference Masyn and Little2013); thus, it is not surprising that most studies that identified four ODD classes also have very large samples (see Supplemental Table).
Some recent longitudinal models have attempted to examine symptom severity across the different dimensions of ODD. Specifically, growth mixture models of the irritable symptoms of ODD in preschool children (ages 3–5) revealed five separate trajectories: high-persistent, decreasing, increasing, low-persistent, and none. Membership in the high-persistent and increasing trajectories predicted poorer functional and psychopathological outcomes at age 6 (Ezpeleta et al., Reference Ezpeleta, Granero, de la Osa, Trepat and Domènech2016). In a particularly compelling set of analyses, Boylan et al. (Reference Boylan, Rowe, Duku, Waldman, Stepp, Hipwell and Burke2017) examined group-based trajectories defined by the three DSM dimensions of ODD. Utilizing data from the Pittsburgh Girls Study (ages 5–13), the authors determined that the defiant/argumentative and antagonism/vindictive dimensions were characterized by low, medium, and high severity while the angry/irritable dimension was best captured by low, medium stable, and high-increasing groups. These findings suggest that the ODD subdimensions are linked to symptom severity and therefore may not represent distinct categories. Continued work is clearly needed here, and longitudinal models spanning multiple developmental periods and multiple datasets may help explain discrepant findings in the literature regarding the dimensions/classes of ODD and how symptom severity fits within these models. Recent work to combine ODD dimensions and severity through factor mixture modeling (Gomez & Stavropoulos, Reference Gomez and Stavropoulos2018) is a particularly promising direction for future research.
Age, sex, and race differences in the ODD classes
Results from our measurement invariance tests suggested important developmental differences in how the latent classes of ODD manifest across adolescence. That is, evidence of full measurement noninvariance or partial measurement invariance suggested that the meaning of the classes was not the same across several time points (Nylund, Reference Nylund2007), a finding that is to be expected when testing measurement invariance longitudinally (Putnick & Bornstein, Reference Putnick and Bornstein2016). Full measurement invariance was only established between grades 7 and 8, suggesting that there may be differences in the latent structure of ODD at several transition points (e.g., moving from elementary to middle school in 6th grade and from middle school to high school in 9th grade), potentially reflecting general shifts in behavior and affect at these developmental transitions. While we identified the same number of classes at all four time points according to both reporters, findings from our measurement invariance tests indicate that the meaning of these classes varies across time. Given that our classes reflect degrees of severity in ODD, future work should explore if different levels of severity in symptom presentation vary in meaning (i.e., that certain ODD symptoms are viewed as more or less severe) at different ages and at specific and important transitional timepoints during adolescence.
As in other studies indicating similarities among males and females in the structure of ODD (Ezpeleta et al., Reference Ezpeleta, Granero, de la Osa, Penelo and Domènech2012; Herzhoff & Tackett, Reference Herzhoff and Tackett2016; Lavigne et al., Reference Lavigne, Gouze, Bryant and Hopkins2014), very few sex differences emerged among our identified classes of ODD. Males were more likely than females to be in the high class at grade 6 (according to adolescent and parent report) and in the medium class at grade 7 (according to parent report). These findings suggest potential male-specific increases in symptoms of ODD during the transition into middle school, as stress related to this developmental transition may lead to higher levels of behavioral and emotional dysregulation among males (Arbuckle & Little, Reference Arbuckle and Little2004). They are also consistent with epidemiological data suggesting that the prevalence of ODD is greater in males than in females across ages 6–16 (Boylan et al., Reference Boylan, Vaillancourt, Boyle and Szatmari2007), and longitudinal data suggesting that ODD symptoms decline for females but remain stable for males across adolescence (Leadbeater et al., Reference Leadbeater, Thompson and Gruppuso2012). However, there is also evidence that this sex difference largely dissipates later in adolescence (Boylan et al., Reference Boylan, Vaillancourt, Boyle and Szatmari2007), and so future studies examining sex differences in the prevalence of ODD across different developmental periods are needed to clarify these findings. Overall though, findings from the broader literature as well as those documented in the current study suggest that ODD remains a concern for some males, particularly in early adolescence and at specific developmental transition points.
Findings regarding racial differences in the latent classes of ODD were largely inconsistent across grades and reporters, making it difficult to draw definitive conclusions. One relatively consistent finding indicated that adolescents in the high and medium classes were more likely to be Black or White as compared to “Other” races (i.e., Asian, Pacific Islander, or Native American) according to both adolescent and parent report. Notably, our findings also highlighted significant differences across reporters, as Black adolescents tended to be in the high and medium classes according to self-report, and White adolescents tended to be in the high and medium classes according to parents. These findings are consistent with empirical work documenting that racial minority adolescents endorse more internalizing and externalizing behaviors than their parents, while White parents report more internalizing and externalizing behaviors than their adolescents (Lau et al., Reference Lau, Garland, Yeh, McCabe, Wood and Hough2004). While the source of these discrepancies is not entirely clear, Lau and colleagues propose that these differences may reflect racial differences in parenting behaviors and/or attitudes toward youth behavior and mental health. Regardless of the source, given that parents are often the “gatekeepers” for whether children and adolescents seek treatment, these racial differences in parental reports of child and adolescent psychopathology may explain some of the disparities in mental health care utilization for racial minority youth (Gudiño et al., Reference Gudiño, Lau, Yeh, McCabe and Hough2009; Lau et al., Reference Lau, Garland, Yeh, McCabe, Wood and Hough2004). Culture affects perceptions of whether behavior is “normative” or “pathological,” which affects decisions about whether or not to seek mental health care. Currently, there is a relative dearth of research on the role of race and ethnicity in informant discrepancies (De Los Reyes et al., Reference De Los Reyes, Augenstein, Wang, Thomas, Drabick, Burgers and Rabinowitz2015; De Los Reyes & Kazdin, Reference De Los Reyes and Kazdin2005). Continued exploration of these differences in informant discrepancies, as they relate to reports of ODD as well as other forms of child and adolescent psychopathology, has broad implications for uncovering racial biases in diagnostic classifications and mental health assessment and treatment.
Links to later psychopathology
When considering our findings regarding the predictive validity to later symptoms of depression and CD, it is important to note that these analyses controlled for baseline levels of these disorders; therefore, these findings highlight the incremental predictive utility of the ODD classes, over and above symptoms of depression and CD that were already present when adolescents entered the study (i.e., grade 6). Despite some inconsistencies in findings across grades and reporters, membership in both the high and medium classes demonstrated concurrent and predictive relations with depression and CD. Furthermore, the high class, as compared to the medium class, relatively consistently predicted higher symptom counts of both depression and CD. Given that the high and medium classes were characterized by endorsement of both the behavioral and affective symptoms of ODD, it is not surprising to see that both of these classes consistently predicted later depression and CD, which may also reflect high levels of comorbidity between internalizing and externalizing disorders (Lilienfeld, Reference Lilienfeld2003). These findings may also be due in part to the fact that these classes (and in particular the medium class) were the most prevalent classes according to both adolescents and parents, with the low class being the least prevalent at all four time points according to parent report. As such, the low class may represent a generally well-adjusted and emotionally and behaviorally stable group of adolescents.
Consistent with work indicating that both the affective and behavioral dimensions/classes of ODD are linked to internalizing and externalizing behaviors (Aebi et al., Reference Aebi, Barra, Bessler, Steinhausen, Walitza and Plattner2016; Drabick & Gadow, Reference Drabick and Gadow2012; Leadbeater & Homel, Reference Leadbeater and Homel2015; Mikolajewski et al., Reference Mikolajewski, Taylor and Iacono2017; Wesselhoeft et al., Reference Wesselhoeft, Stringaris, Sibbersen, Kristensen, Bojesen and Talati2019), our findings indicate that membership in either the high or medium classes predicted both externalizing and internalizing forms of psychopathology up to 3 years later. An important caveat to consider here is that many of these findings were only identified for within-reporter models. Theoretical and empirical work on informant discrepancies suggests that parents and adolescents tend to agree more on ratings of externalizing behaviors and less on internalizing behaviors (De Los Reyes & Kazdin, Reference De Los Reyes and Kazdin2005). Given that ODD encompasses both behavioral and affective dimensions, a crucial direction for future research is to understand how reporter discrepancies manifest within this disorder. Overall, our findings indicate that membership in both the high and medium classes increase risk for various forms of psychopathology later in development, and highlight that ODD may reflect a common thread that explains some of the comorbidity between internalizing and externalizing disorders (Burke & Loeber, Reference Burke and Loeber2010; Burke et al., Reference Burke, Loeber, Lahey and Rathouz2005; Hipwell et al., Reference Hipwell, Stepp, Feng, Burke, Battista, Loeber and Keenan2011). Perhaps the key question here is not whether the different dimensions/classes of ODD differentially predict different forms of psychopathology, but rather what constellation of ODD symptoms, and at what degrees of severity, place youth at heightened risk for later mental health difficulties.
Strengths and Limitations
There are several strengths of the current study, including the use of a longitudinal design with a diverse community-based sample and the application of rigorous and advanced statistical modeling techniques. Given important interrater differences in reports of child and adolescent psychopathology (De Los Reyes & Kazdin, Reference De Los Reyes and Kazdin2005), we collected data from diagnostic interviews with both adolescents and their parents. Similarities in the latent class structure of ODD according to adolescents and parents suggest robustness in the classes across reporters. However, discrepancies in many of our findings depending on the reporter (i.e., prevalence estimates of ODD, sex and race differences in the latent classes and in their prediction to later depression and CD) underscore the importance of collecting data from multiple reporters, particularly given that parent-child informant discrepancies tend to vary depending on race (De Los Reyes et al., Reference De Los Reyes, Augenstein, Wang, Thomas, Drabick, Burgers and Rabinowitz2015; De Los Reyes & Kazdin, Reference De Los Reyes and Kazdin2005; Kim et al., Reference Kim, Chan, McCauley and Vander Stoep2016; Lau et al., Reference Lau, Garland, Yeh, McCabe, Wood and Hough2004). Significant findings were also more frequently observed for within-reporter as compared to cross-reporter models, suggesting that some of our findings may be partly attributable to reporter characteristics, bias, and shared measurement variance. Further consideration of how these informant factors impact on ratings of child and adolescent ODD is warranted.
There are also some limitations to the current study. First, although we used two-component sampling weights to ensure that adolescents in the longitudinal cohort study were comparable to the full screening sample and were representative of middle school students in the Seattle Public School District, it is unclear if similar results would be observed in different populations, cultures, or ages (e.g., clinic-based samples, different geographic areas of U.S., international samples, elementary-aged students). Given some evidence that the latent diagnostic structure of ODD tends to differ depending on whether the sample is community- or clinic-based, and European- or U.S.-based, consideration of these sample-specific characteristics is essential to aid in the interpretation of empirical findings. Additionally, while the current study adds to our understanding of how informant discrepancies may manifest in Black and White parent-adolescent dyads, we had to collapse across other racial and ethnic minority groups (e.g., Latinx, Asian, Native American) for analyses due to small sample sizes. Future research should aim to enroll larger numbers of racial minority youth across all groups, particularly given evidence that parent-adolescent discrepancies in reports of psychopathology tend to vary depending on cultural perspectives and immigrant and acculturation status (Lau et al., Reference Lau, Garland, Yeh, McCabe, Wood and Hough2004). Finally, our longitudinal design allowed us to examine ODD across the middle school years (grades 6–9), but it is unclear if a similar ODD structure would be observed in children or older adolescents.
It is also important to consider evidence of differential attrition in the current study. That is, parents and adolescents identifying as Black, Asian, Pacific Islander, or Native American were more likely to attrit at later time points, further underscoring the need to enhance methods to recruit and retain racial minority families in longitudinal studies. As is commonly observed in other longitudinal studies (e.g., Avon Longitudinal Study of Parents and Children; Wolke et al., Reference Wolke, Waylen, Samara, Steer, Goodman, Ford and Lamberts2009), our missing data analyses also indicated that adolescents with higher baseline symptoms of CD and ODD (according to self-report only) were more likely to attrit at follow-up assessments. Our results therefore may be a more conservative estimate of externalizing behaviors across grades 6–9. However, simulation work (Gustavson et al., Reference Gustavson, von Soest, Karevold and Røysamb2012; Wolke et al., Reference Wolke, Waylen, Samara, Steer, Goodman, Ford and Lamberts2009) indicates that regression estimates are only marginally affected by selective attrition. As such, our findings were likely not significantly affected by this differential attrition across the longitudinal follow-up period.
Clinical implications and conclusions
Our results underscore the importance of developing and testing evidence-based integrated treatments that target comorbid disruptive behavior and depression, particularly modular interventions that can be tailored to transdiagnostic deficits common across different manifestations of adolescent mental health conditions (e.g., self-regulation, emotion socialization; Evans et al., Reference Evans, Wei, Harmon and Weisz2021; Weisz & Bearman, Reference Weisz and Bearman2020; Weisz et al., 2016; Zachary & Jones, Reference Zachary and Jones2019). In fact, recent work by Ollendick et al. (Reference Ollendick, Booker, Ryan and Greene2018) indicated that both Collaborative & Proactive Solutions (a skills-based intervention targeting oppositionality) and Parent Management Training were effective in treating the full range of ODD symptoms (i.e., regardless of dimension). Results from the current study also suggest that the high class displays a particularly severe pattern of ODD, as these adolescents exhibited the highest levels of behavioral and affective difficulties and predicted the highest increases in later symptoms of depression and CD compared to the other classes. There is now a solid research base indicating that youth who experience dysregulation, particularly in terms of emotions, are at heightened risk for later adverse mental health outcomes (McLaughlin et al., Reference McLaughlin, Hatzenbuehler, Mennin and Nolen-Hoeksema2011; Silk et al., Reference Silk, Steinberg and Morris2003). Adolescents in the high class are therefore in great need of targeted interventions aimed at reducing their level of risk for later psychopathology. Future work with person-centered longitudinal models (e.g., latent transition analysis) should also consider whether adolescents in the high class remain in this class or move to lower levels of severity over time. Identifying protective factors that predict adolescents’ transitions to lower severity classes may suggest important targets for intervention.
While the classes in the current study do not support distinct subtypes of ODD, and we cannot conclude that all adolescents in the high class would meet criteria for a clinical diagnosis of ODD, our findings do suggest that this class represents a subgroup of adolescents displaying more severe levels of ODD symptoms. Moving to the next level of severity, the medium class may represent an “at-risk” group of adolescents, but continued work is needed to understand if these class members are displaying psychopathology with functional impairment or, rather, are displaying levels of behavioral and affective dysregulation more common in adolescence (Arnett, Reference Arnett1999; Craig et al., Reference Craig, Hernandez, Moretti and Pepler2021; Silk et al., Reference Silk, Steinberg and Morris2003). In general, more psychometric and diagnostic work is needed to understand the clinical implications of different levels of ODD severity. How clinicians can interpret these classes of ODD severity into meaningful intervention targets to inform clinical practice with adolescents is an important direction for future research, and will likely require the use of clinically focused samples.
The current study is unique in its focus on how ODD manifests across early adolescence. Continued longitudinal work across multiple developmental periods is needed to clarify these findings and those identified in the broader literature, as different diagnostic instruments and treatment approaches may be needed depending on the age of the client. For instance, it may be that with adolescents, more consideration needs to be paid to clients’ unique pattern of behavioral and affective severity in their ODD presentation. There is now a broad and well-supported field of research supporting different dimensions and latent classes within the ODD “umbrella.” A crucial direction for future research is to understand what treatment approaches best serve youth with different symptom presentations, or severity levels, of the same disorder. Examining patterns of item endorsement among children and adolescents falling into each latent class may suggest cutoff values or highly sensitive items on the DISC and other related diagnostic instruments that would more accurately capture different patterns of severity (Lindhiem et al., Reference Lindhiem, Bennett, Hipwell and Pardini2015). Given that the high class is characterized by much higher endorsement of defies, annoys, blames, and spiteful as compared to the other items, these particular symptoms may help differentiate levels of severity in ODD. Consistent with viewpoints supporting the dimensional nature of psychological disorders (Barry et al., Reference Barry, Marcus, Barry and Coccaro2013; Carragher et al., Reference Carragher, Krueger, Eaton and Slade2015), and of ODD in particular (Gomez & Stavropoulos, Reference Gomez and Stavropoulos2018; Roetman et al., Reference Roetman, Siebelink, Vermeiren and Colins2021), it will also be important to determine if these classes of ODD fall along a dimension of severity or whether the classes truly represent distinct clinical phenotypes.
Supplementary material
To view supplementary material for this article, please visit https://doi.org/10.1017/S0954579421001875
Acknowledgements
The authors wish to thank the adolescents and parents who participated in the Developmental Pathways Project as well as the research team. Preliminary versions of this work were presented at the Society for Prevention Research annual meetings, San Francisco, CA, May 2013 and 2016, and the International Society for Research in Child and Adolescent Psychopathology biennial meeting, Leuven, Belgium, June 2013.
Funding statement
This research was supported by grants R01-MH63711 and R01-MH079402 from the National Institute of Mental Health. Other support included Seattle Children’s Hospital Steering Committee Grant and UW Provost Bridge Funding. Additional support for the preparation of this work was provided by a LEEF B.C. Leadership Chair award, B.C. Children’s Hospital Research Institute Investigator Grant Award, and a Canada Foundation for Innovation award to Robert J. McMahon.
Conflicts of interest
None.
