Background
In health care sites in low-resource settings, providers frequently encounter children with physical and cognitive disabilities. Rapid unplanned urbanization, inequitable distribution of resources, fragile health systems, poor obstetric practices, malnutrition and poverty continue to increase the risk of poor child development and child neurodisability in low-resource settings globally (Groce et al., Reference Groce, Challenger, Berman-Bieler, Farkas, Yilmaz, Schultink, Clark, Kaplan and Kerac2014). The focus of the new United Nations (UN) global strategy for women, children and adolescents is to move beyond survival, to ensuring health and wellbeing for everyone and on providing enabling environments (Kuruvilla et al., Reference Kuruvilla, Bustreo, Kuo, Mishra, Taylor, Fogstad, Gupta, Gilmore, Temmerman and Thomas2016). Measuring functioning of an individual in various life domains, in different contexts, and for various states of health and disease is a critical piece of information to inform individualized intervention plans, programmes and health policies (Murray, Reference Murray2002). To materialize this vision, standardized and uniform measurement frameworks such as the ICF (International Classification of Functioning, Disability and Health) (WHO, 2001) and tools based on the ICF such as the WHO Disability Assessment Schedule (WHODAS-2.0) for adults have been developed and validated (Üstün et al., Reference Üstün, Kostanjsek, Chatterji and Rehm2010). WHODAS-2.0 is an open access tool that has been used cross-culturally for various health conditions to provide important health policy information through population surveys (Üstün et al., Reference Üstün, Chatterji, Villanueva, Bendib, Çelik, Sadana, Valentine, Ortiz, Tandon and Salomon2001; Buist-Bouwman et al., Reference Buist-Bouwman, Ormel, De Graaf, Vilagut, Alonso, Van Sonderen and Vollebergh2008; Sousa et al., Reference Sousa, Ferri, Acosta, Albanese, Guerra, Huang, Jacob, Jotheeswaran, Rodriguez and Pichardo2009) and health registries (Gallagher and Mulvany, Reference Gallagher and Mulvany2004). It has also been utilized in formulating and monitoring individualized treatment plans and evaluating effectiveness and cost-effectiveness of public health interventions for adults (Chopra et al., Reference Chopra, Couper and Herrman2004; McKibbin et al., Reference Mckibbin, Patterson and Jeste2004; Chisolm et al., Reference Chisolm, Abrams, Mcardle, Wilson and Doyle2005; Perini et al., Reference Perini, Slade and Andrews2006; Soberg et al., Reference Soberg, Bautz-Holter, Roise and Finset2007; Baron et al., Reference Baron, Schieir, Hudson, Steele, Kolahi, Berkson, Couture, Fitzcharles, Gagné and Garfield2008; Hudson et al., Reference Hudson, Steele, Taillefer and Baron2008; Schlote et al., Reference Schlote, Richter, Wunderlich, Poppendick, Möller and Wallesch2008; Federici et al., Reference Federici, Meloni, Mancini, Lauriola and Olivetti Belardinelli2009).
Many of the child and youth disabilities encountered by health care workers and researchers in low-resource settings fall into the category of neurodevelopmental disorders (Olusanya et al., Reference Olusanya, Davis, Wertlieb, Boo, Nair, Halpern, Kuper, Breinbauer, De Vries and Gladstone2018). The World Health Organization's Mental Health Gap Action Programme (WHO mhGAP) intervention guide (mhGAP-IG) uses the related term ‘Developmental Disorder’ (DD) to define ‘an impairment or delay in functions related to central nervous system maturation impacting multiple developmental domains’ (WHO, 2017). Thus, developmental disorder is an umbrella term covering a range of conditions, including intellectual disability as well as autism spectrum disorders, with the common factor being intellectual and adaptive functioning deficits. The American Psychiatric Association uses the term neurodevelopmental disorder which is defined as, ‘a group of congenital or acquired long-term conditions that are attributed to impairment of the brain and/or neuromuscular system and create functional limitations’ (American Psychiatric Association, 2013). The symptoms of neurodevelopmental disorders may occur in combination with other disorders and the symptoms' severity and complexity may vary over time. Neurodevelopmental disorders may cause difficulties in ‘movement, cognition, hearing and vision, communication, emotion and behaviour’. Common to all neurodevelopmental disorders is the key characteristic of (a) origin in early childhood and (b) impact on multiple developmental domains (motor, cognitive, behavioural and communication) (WHO, 2017). However, no tools like the WHODAS-2.0 exist for children with neurodevelopmental and/or developmental disorders that are open source, simple to administer, and valid over a range of child development domains, age groups and socio-cultural settings.
Traditional assessment of childhood developmental disorders either involves screening by using brief, valid questionnaires such as Ten Questions Screening Tool (Durkin et al., Reference Durkin, Wang, Shrout, Zaman, Hasan, Desai and Davidson1995b) to identify children with or without disability or detailed developmental assessment using standardized tools by a multidisciplinary team of experts for diagnosis and evaluation of adaptive functioning in children. The brief screening tools are reliable and valid for screening purpose only, i.e. they fail to measure the impact of child disability on child functioning. The standard tools for assessment are often copyrighted, need extensive training and supervision for administration, scoring and interpretation, and thus are not available in low-resource settings or practical in pragmatic research.
One reason why more simple tools for children can be difficult to develop is that children's functioning is so highly context and culturally specific (Scorza et al., Reference Scorza, Stevenson, Canino, Mushashi, Kanyanganzi, Munyanah and Betancourt2013). The WHO's ICF-CY (International Classification of Functioning-Child and Youth version) provides a common terminology for identifying functional problems in children and adolescents, including bodily functioning, activity limitations and participation restrictions (Scorza et al., Reference Scorza, Stevenson, Canino, Mushashi, Kanyanganzi, Munyanah and Betancourt2013), but there has been limited progress towards development and validation of measurement tools which are grounded in this potentially trans-cultural framework. One group has developed and validated a youth version of WHODAS 2.0 for adolescents (aged 10–17) with mental health problems in rural Rwanda (Scorza et al., Reference Scorza, Stevenson, Canino, Mushashi, Kanyanganzi, Munyanah and Betancourt2013) and named it WHODAS-Child. The tool demonstrated good internal consistency (α = 0.83) and reliability in measuring functional impairment among adolescents with socio-emotional problems, but there remains a need for validation of a tool suitable for younger children with developmental disorders.
In this paper, we describe the validation of WHODAS-Child (Scorza et al., Reference Scorza, Stevenson, Canino, Mushashi, Kanyanganzi, Munyanah and Betancourt2013) for use with children with developmental disorders in Pakistan. We followed the WHO mhGAP-IG definition of development disorders, ‘development disorder is an umbrella term covering disorders such as intellectual disability as well as autism spectrum disorders. These disorders usually have a childhood onset, impairment or delay in functions related to central nervous system maturation, and a steady course rather than the remissions and relapses that tend to characterize many other mental disorders’ (WHO, 2017). This paper describes the results of factor analysis, reliability, validity and sensitivity to change analyses of the adapted WHODAS-Child in rural Pakistan.
Methods
Study settings
We conducted the study in the rural sub-district (Gujar Khan) of Rawalpindi, Pakistan (estimated population of one million). In Pakistan, the smallest administrative unit in a sub-district is a Union Council (UC). Each UC has about 12–15 villages with a population of approximately 30 000–50 000. Each UC has a Primary Health Care (PHC) facility called a Basic Health Unit (BHU). The BHU has a work force of a primary care physician or a medical technician, a Lady Health Visitor (LHV), a vaccinator and a set of Lady Health Workers (LHWs). The study was conducted in 30 UCs of sub-district Gujar Khan. The infant mortality rate in Rawalpindi has been estimated to be 55 per thousand live births, the under-5 mortality rate is 82 per thousand live births and the percentage of underweight children below 5 years of age is 25% (Bureau of Statistics & UNICEF Pakistan, 2003). In a previous study, 34.2% of children in Rawalpindi district were screened positive on the Ten Questions Screen (TQS) for childhood developmental difficulties (Mirza et al., Reference Mirza, Mehmood, Tareen, Davidson and Rahman2008).
Sampling
The data for this validation study were collected as a part of an implementation research trial embedded within the scaled-up implementation of WHO mhGAP-based Parent Skills Training (PST) programme in rural Pakistan (Hamdani et al., Reference Hamdani, Akhtar, Huma, Nazir, Minhas, Sikander, Wang, Servilli and Rahman2017). The host organization has a database of about 3000 families and children with developmental disorders as a part of the host organization's service delivery to the community (Hamdani et al., Reference Hamdani, Minhas, Iqbal and Rahman2015). For the purpose of the cluster randomized controlled trial (cRCT), a sample size of 540 parent–child dyads from 30 clusters (18 parent–child dyads per cluster) was required to evaluate the impact of PST programme (Hamdani et al., Reference Hamdani, Akhtar, Huma, Nazir, Minhas, Sikander, Wang, Servilli and Rahman2017). To ensure the representativeness of the study sample, evaluation zones were created within each UC. These evaluation zones were made by randomly selecting one LHW out of the total 15–20 LHWs serving in a particular UC. The random selection was carried out by an independent researcher using a simple random table. If the required number of dyads was not completed from one LHW catchment area, the adjacent LHW catchment area was included to reach the required sample size from each UC. This patch of contiguous area/catchment areas of LHWs within that UC served as the impact evaluation zone for the trial-related outcomes. This sampling process ensured that 540 parent–child dyads (18 parent–child dyads per cluster) are randomly selected from 3000 families and children participating in the scaled-up implementation of programme. For the purpose of validation study, a sub-set of the data consisting of 400 parent–child dyads was randomly selected using Statistical Package for Social Sciences (SPSS).
Participants
The target population for this study was the parents/caregivers of children with developmental disorders aged 2–12 years. The study sample (N = 800) involved 400 parents of children with developmental disorders and 400 parents of age- and gender-matched children without developmental disorders from the same community.
Our approach to identify children with development disorders in rural community settings is informed by a similar study in rural India by Singhi et al. (Reference Singhi, Kumar, Malhi and Kumar2007). In phase 1, TQS was administered to parents of children aged between 2 and 12 years. In phase 2, all children who screened positive were clinically evaluated in detail following the mhGAP guidelines by a trained psychologist.
Children with developmental disorders
Inclusion criteria
• Children who screened positive on any of the TQS questionnaire items # 1, 4, 5, 7, 8, 9, 10 for developmental delay (Durkin et al., Reference Durkin, Hasan and Hasan1995a).
• Identified as developmental disorder in the child, according to clinical assessment (history and clinical examination for developmental delay in motor, communication, social, cognitive, daily living skills domains according to WHO mhGAP developmental disorders guidelines for clinical assessment in primary healthcare settings).
Exclusion criteria
Exclusion criteria include
• Deafness or blindness in the child or caregiver.
• Primary caregiver not available or unwilling to participate in the study.
• Primary caregiver unable to speak or understand the Urdu language.
Children without developmental disorders
Inclusion criteria
• Primary caregiver of children aged 2–12 years, who screened negative for developmental disorders using TQS.
• Residing in the study area.
Exclusion criterion
• Inability to speak or understand the Urdu language.
The Nquery Advisor (Elashoff, Reference Elashoff2014) was used to estimate the sample size for test–retest analysis, inter-rater reliability analysis, discriminative reliability and convergent validity. To determine the sufficiency of sample size to estimate a fit for the model, we used the ratio of 10 participants (N) for 1 parameter (p). This is consistent with sample size calculations for Confirmatory Factor Analysis (CFA) in the Rwanda study where with a sample size of 367, the data set met minimum sample guidelines for the CFA (MacCallum et al., Reference Maccallum, Widaman, Zhang and Hong1999).
Data collection procedures
The current study was embedded within a scaled-up implementation of a PST programme, based on guidance from WHO's mhGAP-IG (WHO, 2017) and using core contents (‘key messages’ and strategies) of the WHO Caregivers Skills Training (CST) programme (Salomone et al., Reference Salomone, Pacione, Shire, Brown, Reichow and Servili2019). Data collection took place over a period of 6 months (Hamdani et al., Reference Hamdani, Akhtar, Huma, Nazir, Minhas, Sikander, Wang, Servilli and Rahman2017). To overcome the barrier of illiteracy, all instruments were administered by trained interviewers to the parents/caregivers. Non-specialist health care workers were trained prior to data collection and those with good inter-data collector reliability scores ICC >0.80 (Koo and Li, Reference Koo and Li2016) were selected to collect data. Data were collected electronically using handheld devices on an android-based application named Open Data Kit (ODK)Footnote 1Footnote † and checked periodically for consistency and completeness.
Ethics approval for the study was obtained from the Ethics Review Committee of the Human Development Research Foundation, Islamabad, Pakistan. Parental written informed consent was obtained from all participants, either by written signature or by finger prints depending upon literacy level of the parents/caregivers. All participating children had access to primary healthcare centres and specialist child mental health unit at the Institute of Psychiatry, WHO Collaborating Centre for mental health research and training, Rawalpindi (WHO-CC) (the tertiary mental health care facility in the study sub-district).
Translation and adaptation of WHODAS-Child for children with developmental disorders (henceforth called Developmental Disorders-Children Disability Assessment Schedule – DD-CDAS)
We used a mixed-methods approach to translate and culturally adapt the WHODAS-Child into Urdu for administration by an interviewer. Interviews were conducted with parents of children with and without developmental disorders to capture local norms of age-appropriate healthy functioning in children through free list interviews (Van Ommeren et al., Reference Van Ommeren, Sharma, Thapa, Makaju, Prasain, Bhattarai and De Jong1999). We then made general adaptations such as (a) rephrasing items to target parents/caregivers of children as respondents, (b) making each item more comprehensible for parents with a common stem added to each item and (c) providing culturally specific examples of child functioning based upon the free list interviews. Details of the qualitative study with the free list interviews as well as the translation and adaptation process are reported elsewhere (Hamdani et al., forthcoming). The initial working version of the adapted WHODAS-Child for children with developmental disorders (DD-CDAS), prior to further validation, consisted of 36 items on functioning and disability covering six domains with a recall period of 30 days. The distribution of items in the six domains was as follows: understanding and communication (six items), getting around (five items), self-care (four items), getting along with others (five items), life activities (four items related to household activities and five items for work/school) and participation in society (seven items). The 36 items were rated on a five-point scale where 1 = none, 2 = mild, 3 = moderate, 4 = severe, 5 = extreme/cannot do. Using the standard scoring algorithm from WHODAS-Child, school-related items were not rated for children who were not attending the school and life activities domain and total disability scores were computed from the remaining items (Üstün et al., Reference Üstün, Kostanjsek, Chatterji and Rehm2010). Domain and total raw scores were transformed into a range from 0 to 100. A global disability score was computed from all 36 items. A higher score indicates greater disability or difficulty in functioning. DD-CDAS items along with their corresponding ICF-CY codes are given as an Additional file A.
Criterion validity was assessed by correlating DD-CDAS global disability score with clinician-rated Developmental Disabilities Children's Global Assessment Scale (DD-CGAS) scores (Shaffer et al., Reference Shaffer, Gould, Brasic, Ambrosini, Fisher, Bird and Aluwahlia1983). The DD-CGAS rating is a global rating of functioning in children based on all available sources of information and across all domains of functioning, including self-care, communication, social behaviour and school/academic functioning (Wagner et al., Reference Wagner, Lecavalier, Arnold, Aman, Scahill, Stigler, Johnson, Mcdougle and Vitiello2007; Olsson and Bölte, Reference Olsson and Bölte2014; White et al., Reference White, Smith and Schry2014). A high inverse correlation was expected as the DD-CDAS measures the construct of difficulty in functioning whereas DD-CGAS is a measure of global functioning.
Convergent validity: We used the Vineland Adaptive Behaviour Composite (VABS-II-ABC score) (Sparrow et al., Reference Sparrow, Cicchetti and Balla2005) and domain scores to assess the convergent validity of DD-CDAS. The VABS-II-ABC score measures the adaptive functioning of individuals from birth to 90 years of age. VABS-II consists of five domains with each having 2–3 sub-domains. The primary domains are communication, daily living skills, socialization and motor skills. A total sum of all domains produces an Adaptive Behaviour Composite score. A higher score indicates better functioning. An adapted version of VABS-II for children with developmental disorders was used in the present study (Rahman et al., Reference Rahman, Divan, Hamdani, Vajaratkar, Taylor, Leadbitter, Aldred, Minhas, Cardozo and Emsley2016). Data to assess the convergent validity was obtained from a smaller sub-sample (N = 68).
We did a correlation analysis between the pro-social domain scores of Strengths and Difficulties Questionnaire (SDQ) (Goodman, Reference Goodman1997) and functional disability of DD-CDAS in children with developmental disorders. The pro-social domain of SDQ measures how much a child is considerate of other people's feelings; shares readily; helpful if someone is hurt. The higher the score, the more pro-social a child is. We used the parent-rated version of the tool which has been validated in a similar population (Maselko et al., Reference Maselko, Sikander, Bangash, Bhalotra, Franz, Ganga, Rajan, O'donnell and Rahman2016).
Discriminative validity of DD-CDAS was assessed by comparing the total score and domains score of age- and gender-matched children with developmental disorders with the control sample of children without developmental disorders. The total difficulty scores of children without developmental disorders were expected to be better (lower) than children with developmental disorders.
Subgroup analysis was conducted to evaluate DD-CDAS ability to differentiate among members of the sample with different developmental disorders. Descriptive statistics were calculated to compare domains and global disability scores across groups.
Reliability was assessed through measuring internal consistency, test–retest (reproducibility) and inter-data collector reliability (where only the data collector was changed, not the respondent). Internal consistency was evaluated with the Cronbach's α coefficients. To assess test–retest, a sub-sample of participants was identified and assessed at 2-week intervals by the same rater. As the sample consisted of children with developmental disorders, their clinical severity was not expected to change over a period of 2 weeks. Inter-data collector reliability was assessed by administering the DD-CDAS to the same caregivers by two different non-specialist health care providers. Concordance in the scores was estimated by calculating the One-way Random Intra-class Correlation Coefficient (ICC).
Sensitivity to change: The responsiveness of DD-CDAS was evaluated by administering it 6 months after pre-test. The study sample consisted of caregivers of children with developmental disorders, who received training in the PST programme (Hamdani et al., Reference Hamdani, Akhtar, Huma, Nazir, Minhas, Sikander, Wang, Servilli and Rahman2017).
CFA of DD-CDAS: CFA was performed to assess the hypothesized six-domain structure of the DD-CDAS, i.e. understanding and communication (cognition), getting around (mobility), self-care, getting along with people, life activities and participation in society. Goodness-of-fit was measured by the Root Mean Square Error of Approximation (RMSEA, adequate if below 0.08), Comparative Fit Index (CFI), Incremental Fit Index (IFI) and Tucker–Lewis Index (TLI), which are recommended to be over 0.90 (Hooper et al., Reference Hooper, Coughlan and Mullen2008). The relative/normed χ2/df (Wheaton et al., Reference Wheaton, Muthen, Alwin and Summers1977) was used to minimize the impact of sample size on model fit (Hooper et al., Reference Hooper, Coughlan and Mullen2008). CFA was conducted using AMOS version 21.0 with maximum likelihood estimation. Missing values were considered missing at random and were handled by using maximum likelihood estimation due to the very small number of missing values in the study data.
Results
Descriptive statistics for DD-CDAS global disability scores and domain scores were examined for children with and without developmental disorders.
Sample characteristics
Table 1 shows sample characteristics. In the developmental disorders sample, the majority of children were male (62%); 37% of the participants were aged between 2 and 5 years, 33% were aged between 6 and 8 years and 30% were aged between 9 and 12 years. Among them, most of the children were not attending schools (69%). In 96% of cases, mothers were the primary caregiver. There was a significant difference between both groups in terms of gender distribution, family structure, history of developmental disorder in family and number of children attending schools. Only 1% sample of the children with developmental disorders were attending special school.
Table 1. Demographic characteristics of study participants (N = 800)
TQS , Ten Questions Screening.
Descriptive statistics
The average interview time based on an informal estimation of data collectors to administer DD-CDAS was 20 minutes. The distribution of DD-CDAS scores for children with and without developmental disorders is reported in Table 2 and distribution of DD-CDAS scores for children with developmental disorders in Fig. 1. The mean global disability score for children with developmental disorders was 52.00 (s.d. = 21.98) ranging from 4.17 to 97.92, while for children without developmental disorders, the mean global disability score was 2.14 (s.d. = 4.00) ranging from 0 to 26.06. In children with developmental disabilities, no significant differences were observed between girls and boys on global disability score [51.87 (s.d. = 22.06) v. 52.21 (s.d. = 21.88); 95% CI −4.80 to 4.10]. Similarly, no significant differences were observed in sub-domain scores between boys and girls. The normal distribution curves with a histogram for ‘global disability score’ of DD-CDAS for children with and without developmental disorders are mentioned in Fig. 2.
Fig. 1. Developmental Disorders-Children Disability Assessment Schedule (DD-CDAS) domain profile by sub-group (N=400) (N = 800).
Fig. 2. Distribution curve for ‘global disability score’ of DD-CDAS for children with and without developmental disorders (N = 800).
Table 2. DD-CDAS domains profile by sub-group (N = 800)
DD-CDAS, Developmental Disorders-Children Disability Assessment Schedule; DD-CGAS, Developmental Disabilities Children's Global Assessment Scale.
a Data of children with developmental disorders who attended the school (31%) is included in analysis only.
Confirmatory Factor Analysis
CFA was conducted on the data collected from caregivers of children with developmental disorders. As the majority of these children were not attending school, we conducted factor analysis (and all other analyses) on 31 items excluding five school-related items. CFA confirmed that the six-factor model structure of the DD-CDAS is consistent with the structure of the original WHODAS-Child. The values of the RMSEA [χ2/df 2.86, RMSEA 0.068 (90% CI 0.064–0.073), TLI 0.92, CFI 0.93 and IFI 0.93] (Fig. 3) indicated good fit for the six-factor model (Scorza et al., Reference Scorza, Stevenson, Canino, Mushashi, Kanyanganzi, Munyanah and Betancourt2013), also indicating acceptable model fit (Hooper et al., Reference Hooper, Coughlan and Mullen2008; Jackson et al., Reference Jackson, Gillaspy and Purc-Stephenson2009) with all items loading to their respective domains. Only one item in the domain of participation in society, ‘In your opinion, to what extent did your child not get opportunities to take part in social activities the way they ought to have been given to him/her?’, had a factor loading of 0.34. Second-order CFA confirmed a two-level hierarchical structure, with one global disability factor feeding into the six domains [χ2/df 2.70, RMSEA 0.065 (90% CI 0.061–0.070), TLI 0.93, CFI 0.93 and IFI 0.93] (Fig. 4).
Fig. 3. First-order Confirmatory Factor Analysis of the Developmental Disorders-Children Disability Assessment Schedule (DD-CDAS).
Fig. 4. Second-order Confirmatory Factor Analysis of Developmental Disorders-Children Disability Assessment Schedule (DD-CDAS).
Criterion validity
A high inverse correlation of (r = −0.70, p < 0.01) was observed between DD-CGAS and DD-CDAS, indicating that both scales measure similar construct in opposite ways; one measuring functioning and other measuring functioning impairment or disability.
Convergent validity
We found moderate to high correlation coefficients between DD-CDAS in comparison with the Vineland Adaptive Behaviour Scale II (VABS II) (Table 3). The constructs of the DD-CDAS correlated well with similar constructs of VABS II. For example, the DD-CDAS constructs of ‘understanding and communication’ were well correlated with ‘daily living skills (−0.63) and communication (−0.60)’ of the VABS II and the DD-CDAS construct of ‘self-care’ was well correlated with ‘adaptive behaviour (−0.60) and daily living skills (−0.64)’ constructs of VABS II. Most of the other coefficients ranged between 0.43 and 0.77, except 0.35 between participation in society and socialization domains.
Table 3. Correlation coefficients between Vineland Adaptive Behaviour Scales II (VABS-II) and DD-CDAS (N = 68)
DD-CDAS, Developmental Disorders-Children Disability Assessment Schedule.
**p < 0.01.
Table 4 describes the correlation of DD-CDAS with the pro-social behaviour subscale of SDQ. The moderate negative correlation (r = −0.57, p < 0.01) between the two measures highlights that the children with DDs have difficulties with prosocial behaviour (Russell et al., Reference Russell, Rodgers and Ford2013).
Table 4. Correlation analysis of DD-CDAS with pro-social construct of SDQ (N = 400)
DD-CDAS-DDs, Developmental Disorders-Children Disability Assessment Schedule; SDQ, Strengths and Difficulties Questionnaire.
**p < 0.01.
Discriminative validity
We observed significant differences between children with and without developmental disorders on the DD-CDAS global disability mean score and the DD-CDAS domain subscales (Table 2) We found that the mean scores of children with developmental disorders were high on all domains of the DD-CDAS as compared to children without developmental disorders, reflecting that children with developmental disorders are significantly more likely to experience problems in managing life activities in comparison to children without developmental disorders.
Internal consistency
The internal consistency between DD-CDAS items was high. Cronbach's α coefficients for the different DD-CDAS domains were as follows: understanding and communication (six items), 0.85; getting around (five items), 0.92; self-care (four items), 0.84; getting along with people (five items), 0.92; life activities – household (four items), 0.94; life activities – school (five items), 0.92; and participation in society (seven items), 0.82. Total internal consistency of the DD-CDAS was 0.91 for 36 items and 0.95 for 31 items excluding school-related items.
Test–retest reliability
The DD-CDAS showed good test–rest reliability. The ICC ranged from 0.83 to 0.99 for domains, and 0.98 for overall score. All correlations were significant at 0.01 level.
Inter-data collector reliability
The results of the two observations for the same case by two assessors were correlated to evaluate the inter-data collector reliability of disability assessment schedule. The ICCs for the inter-data collector reliability of DD-CDAS ranged from 0.97 to 0.99 (p < 0.01).
Subgroup analysis
(A) Age groups (2–5, 6–8 and 9–12 years)
The mean global disability score in children aged 2–5 years was 55.23 (s.d. = 23.68), compared to 52.19 among children aged 6–8 years (s.d. = 22.02), while for children aged 9–12 years, the mean global disability score was 47.51 (s.d. = 18.81). There was a statistically significant difference among the three age groups on global disability score (F = 4.46, p = 0.021); and the getting around (F = 8.84, p < 0.021), self-care (F = 9.69, p = 0.01) domains of DD-CDAS as determined by one-way ANOVA (see Table 5).
Table 5. Age differences in DD-CDAS scores (N = 400)
*DD-CDAS, Developmental Disorders-Children Disability Assessment Schedule.
Figure 5 shows that DD-CDAS domain scores for children with specific developmental difficulties were in line with their disabilities. Children with motor difficulties (n = 30) had higher disability scores on ‘getting around’ [68.33 (s.d. 29.86)] and ‘life activities’ [67.56 (s.d. = 27.18)] domains, while children with cognitive impairment with motor difficulties (n = 110) exhibited the highest level of overall disability in the sample [71 (s.d. = 20.03)] as well as in all the sub-domains.
Fig. 5. DD-CDAS domain profile by sub-group (N = 400).
Sensitivity to change
Differences in pre- and post-intervention scores were computed using paired sample t test. Small but significant differences were observed at post-intervention in global disability score (ES = 0.19), understanding and communication (ES = 0.17), self-care (ES = 0.22) and life activities scores (ES = 0.23), indicating that DD-CDAS is sensitive to small changes in functioning (see Table 6).
Table 6. DD-CDAS sensitivity to change analysis (N = 246)
DD-CDAS, Developmental Disorders-Children Disability Assessment Schedule.
Note: The post-test data were collected after 6-month pre-test.
Discussion
The study results provide evidence that the DD-CDAS satisfies many of the criteria for a pragmatic measurement tool and the data obtained from the implementation of DD-CDAS can be used to answer real-world questions to scale-up of programme for children with developmental disorders in low-resource settings. After only 2 days of training, lay health workers were able to administer the DD-CDAS to caregivers of children with DDs. Administration time was relatively short and the instrument showed good psychometric properties in children with developmental disorders, including high reliability, responsive to the functional disability of children with different developmental difficulties and adequate sensitivity to change over time following treatment.
To rigorously evaluate the factor structure of DD-CDAS, a combination of fit indices was used. Most items fitted in their theoretically assigned domains. The CFA of DD-CDAS showed a rigorous association between items and domains of DD-CDAS. Although each domain of DD-CDAS is unidimensional, given the correlations between domains, they produced a global disability factor. Only one item in the ‘participation in society’ domain exhibited low factor loading and is recommended to be removed from the final version of the tool. Overall, the results indicated that DD-CDAS domains were able to capture disability across the ICF-CY domains and across the various age and developmental conditions-related sub-groups.
While the findings of this study need to be replicated in other settings, the study has several implications for the field of global mental health. The strong inverse correlation between the parent-rated DD-CDAS and specialist-rated DD-CGAS scores strengthens the argument that functioning can be reliably rated by non-specialists. In addition, the DD-CDAS maps well onto the ICF-CY model of childhood functional disability that emphasizes the role of environment and person–environment interaction in determining functional outcomes for children with developmental disorders. It can thus help guide holistic treatment that targets both a child's condition and the contextual factors that influence functioning (Gladstone et al., Reference Gladstone, Lancaster, Umar, Nyirenda, Kayira, Van Dan Broek and Smyth2010; Charman and Gotham, Reference Charman and Gotham2013). Since the DD-CDAS is open access easy to administer in low-resource settings by non-specialist health care workers and sensitive to change, it has the potential to inform the real-world implementation of truly bio-psycho-social interventions for childhood developmental disorders, at-scale in low-resource community settings.
Our results are remarkably similar to the studies conducted using the adult version of the WHODAS-2.0 across multiple cultural settings (Federici et al., Reference Federici, Bracalenti, Meloni and Luciano2017) and a WHODAS-Child validation study in rural Rwanda (Scorza et al., Reference Scorza, Stevenson, Canino, Mushashi, Kanyanganzi, Munyanah and Betancourt2013) that assessed children with mental health problems. In the adaptation of WHODAS-Child developed in Rwanda, the domain structure and overall structure of the tool were similarly preserved compared to the original instrument. We added culturally appropriate examples of local functioning to adapt the individual items in the WHODAS-Child to suit the context. Together, these studies suggest that the overall structure of the WHODAS-Child and DD-CDAS represents common domains of functional disability across mental health and developmental conditions for populations in rural Rwanda and rural Pakistan. This is an important attribute to allow cross-population comparability of the research data to inform global health policy and practice.
Sub-group analysis of DD-CDAS scores, according to three age groups 2–5, 6–8 and 9–12 years, revealed statistically significant differences in global disability scores and domain scores in getting around and self-care domains. The scores were higher for children in younger age groups (aged 2–5 and 6–8 years) as compared to children 9–12 years old.
The findings of this study also contribute to a wider academic discourse on the role of functioning in ICD-10 and DSM-V. The Axis VI of the Multiaxial System of Classification of Child and Adolescent Psychiatric Disorders in ICD-10 recommends the use of ‘Global Assessment of Psychosocial Disability’ (GAPD). The tool has been criticized due to insufficient reliability in clinical settings, lack of precision, inability to detect change and limited evidence of concurrent validity (Schorre and Vandvik, Reference Schorre and Vandvik2004). To address this critique, there is an initiative from the WHO to recommend the use of WHODAS-2.0 along with the ICD-11 diagnostic categories (Selb et al., Reference Selb, Kohler, Nicol, Riberto, Stucki, Kennedy and Üstün2015). For adults, the American Psychological Association (APA) now suggests that clinicians use the WHODAS-2.0 as a measure of disability (Konecky et al., Reference Konecky, Meyer, Marx, Kimbrel and Morissette2014). It is likely that childhood conditions in these classification systems will follow the suit (Glorisa et al., Reference Glorisa, Prudence, Margarita and Hector2013). The findings from this study will support these moves, and contribute to moving the field of measurement forward in a substantial way.
Limitations
These results need to be interpreted with caution since the DD-CDAS was not specifically adapted for the youngest age group (2–5 years) and this sub-sample was not powered to detect statistically significant differences between age groups (though one was found). Further studies with clinical evaluation of the study sample for each of the 2–5, 6–8 and 9–12 age groups are recommended. The sensitivity to change data of DD-CDAS needs to be interpreted with caution as there was no control group for the intervention part of this study and neither did we use any additional measure to assess whether the change at 6 months after pre-test was real and meaningful. Also as the observed effect sizes are small, any RCT will need a larger sample size if DD-CDAS is used as the primary outcome measure. To enhance its utility in clinical settings, further studies with a clinician-administered version of DD-CDAS are needed. Comparison of caregiver- and clinician-rated DD-CDAS can provide useful insights to develop management plans for children with developmental disorders.
Conclusion
This research shows that DD-CDAS has the potential to be an easy to administer, reliable and valid tool to measure functional disability in children with developmental disorders in low-resource settings. It has the potential to be integrated into global child health programmes to provide data on an individual level, as well as to provide data for comparison across programmes, developmental conditions, cultures and delivery systems in a sustainable and scalable manner. However, further studies are warranted to assess psychometric properties of the tool in different age groups, contexts and for different developmental conditions to validate these findings, and also further evaluate its sensitivity to change over a longer period of follow-up and compared to a typically developing child.
Supplementary material
The supplementary material for this article can be found at https://doi.org/10.1017/gmh.2020.10
Acknowledgements
We are grateful to all the families who participated in the study, Lady Health Workers and research staff (Mr Muhammad Aslam, Ms Parveen Akhter, Ms Asia Khan, Ms Huma Nazir, Ms Mahjabeen Tariq and Ms Ahmareen Kokab) for their support in the data collection.
Financial support
The adaptation and validation of the WHODAS-Child was a part of a scale-up research project, supported by Grand Challenges Canada (Grant #0746-05-GMH). Grand Challenges Canada is funded by the Government of Canada and is dedicated to supporting Bold Ideas with Big Impact® in global health. Matching funding was provided by Autism Speaks, USA (Grant # 1037) and the Human Development Research Foundation, Islamabad, Pakistan. The funders had no role in study design or conduct, analysis and interpretation of data, or writing of the manuscript.
Conflict of interests
The authors declare that they have no competing interests.
Ethical standards
The authors assert that all procedures contributing to this work comply with the ethical standards of the relevant national and institutional committees on human experimentation and with the Helsinki Declaration of 1975, as revised in 2008. Ethics approval for the study was obtained from the Ethics Review Committee of the Human Development Research Foundation, Islamabad, Pakistan. Informed consent was obtained from each participant.
Open access
