Hostname: page-component-848d4c4894-cjp7w Total loading time: 0 Render date: 2024-06-24T13:48:13.689Z Has data issue: false hasContentIssue false

Cost–utility analysis of Social Stories™ for children with autism spectrum disorder in mainstream primary schools: results from a randomised controlled trial

Published online by Cambridge University Press:  03 June 2024

Han-I. Wang*
Affiliation:
Department of Health Sciences, University of York, UK; and York Trial Unit, Department of Health Sciences, University of York, UK
Kerry Bell
Affiliation:
Department of Health Sciences, University of York, UK; and York Trial Unit, Department of Health Sciences, University of York, UK
Jane Blackwell
Affiliation:
Department of Health Sciences, University of York, UK; and York Trial Unit, Department of Health Sciences, University of York, UK
Charlie Welch
Affiliation:
Department of Health Sciences, University of York, UK; and York Trial Unit, Department of Health Sciences, University of York, UK
Laura Mandefield
Affiliation:
Department of Health Sciences, University of York, UK; and York Trial Unit, Department of Health Sciences, University of York, UK
Judith Watson
Affiliation:
Department of Health Sciences, University of York, UK; and York Trial Unit, Department of Health Sciences, University of York, UK
Emma Standley
Affiliation:
Department of Health Sciences, University of York, UK; and York Trial Unit, Department of Health Sciences, University of York, UK
Dean McMillan
Affiliation:
Department of Health Sciences, University of York, UK; and Hull York Medical School, University of York, UK
Simon Gilbody
Affiliation:
Department of Health Sciences, University of York, UK; and Hull York Medical School, University of York, UK
Barry Wright
Affiliation:
Department of Health Sciences, University of York, UK; Hull York Medical School, University of York, UK; and Child Oriented Mental Health Intervention Centre (COMIC), Leeds and York Partnership NHS Foundation Trust, York, UK
Catherine Hewitt
Affiliation:
Department of Health Sciences, University of York, UK; and York Trial Unit, Department of Health Sciences, University of York, UK
Steve Parrott
Affiliation:
Department of Health Sciences, University of York, UK; and York Trial Unit, Department of Health Sciences, University of York, UK
*
Correspondence: Han-I. Wang. Email: [email protected]
Rights & Permissions [Opens in a new window]

Abstract

Background

One in 57 children are diagnosed with autism in the UK, and the estimated cost for supporting these children in education is substantial. Social Stories™ is a promising and widely used intervention for supporting children with autism in schools and families. It is believed that Social Stories™ can provide meaningful social information to children that can improve social understanding and may reduce anxiety. However, no economic evaluation of Social Stories has been conducted.

Aims

To assess the cost-effectiveness of Social Stories through Autism Spectrum Social Stories in Schools Trial 2, a multi-site, pragmatic, cluster-randomised controlled trial.

Method

Children with autism who were aged 4–11 years were recruited and randomised (N = 249). Costs measured from the societal perspective and quality-adjusted life-years (QALYs) measured by the EQ-5D-Y-3L proxy were collected at baseline and at 6-month follow-up for primary analysis. The incremental cost-effectiveness ratio was calculated, and the uncertainty around incremental cost-effectiveness ratios was captured by non-parametric bootstrapping. Sensitivity analyses were performed to evaluate the robustness of the primary findings.

Results

Social Stories is likely to result in a small cost savings (–£191 per child, 95% CI −767.7 to 337.7) and maintain similar QALY improvements compared with usual care. The probability of Social Stories being a preferred option is 75% if society is willing to pay £20 000 per QALY gained. The sensitivity analysis results aligned with the main study outcomes.

Conclusions

Compared with usual care, Social Stories did not lead to an increase in costs and maintained similar QALY improvements for primary-aged children with autism.

Type
Paper
Creative Commons
Creative Common License - CCCreative Common License - BY
This is an Open Access article, distributed under the terms of the Creative Commons Attribution licence (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted re-use, distribution and reproduction, provided the original article is properly cited.
Copyright
Copyright © The Author(s), 2024. Published by Cambridge University Press on behalf of Royal College of Psychiatrists

Autism spectrum disorder is a lifelong neurodevelopmental condition that includes differences in the way children experience the world, such as social communication differences that can lead to stress in interactions.1 In the UK, one in 57 children are diagnosed with autism,Reference Roman-Urrestarazu, van Kessel, Allison, Matthews, Brayne and Baron-Cohen2 and the estimated cost of supporting these children is around £3.1–3.4 billion (in 2011 value) per year, with special educational needs being the main cost driver (47%), followed by reported parental productivity loss as they care for their children (12%).Reference Buescher, Cidav, Knapp and Mandell3

Given this significant cost in education systems and limited funding available for specialist support in schools,Reference Lindsay, Proulx, Thomson and Scott4 it is important that clinically effective interventions that are also cost-effective can be delivered within schools on a day-to-day basis; that they are safe and child-centred; and that they are tailored to a neurodiverse population. Currently, only one relevant economic evaluation study was done;Reference Wang, Wright, Bursnall, Cooper, Kingsley and Couteur5 the study suggests that the LEGO®-based therapy delivered in a school setting is likely to lead to a small cost saving and small improvements in quality-adjusted life-years (QALYs) compared with usual care.

Study aims

Carol Gray's Social Stories™ is a promising intervention that can potentially alleviate the social communication difficulties,Reference Kokina and Kern6,Reference Wright, Marshall, Adamson, Ainsworth, Ali and Allgar8 as well as not being costly, intrusive, time-consuming or requiring extensive involvement of outside experts.Reference William and Wright9 Such positive outcomes and features make the use of Social Stories very popular in schools and in families with children with autism.Reference Sansosti, Powell-Smith and Kincaid10 Despite the growing interest, there is a lack of economic evaluation study for Social Stories. Therefore, this study aims to assess the cost-effectiveness of Social Stories alongside usual care for children with autism in primary schools, compared with usual care alone. This paper reports the economic evaluation results of Social Stories alongside the Autism Spectrum Social Stories in Schools Trial 2 (ASSSIST-2) trial, and follows the Consolidated Health Economic Evaluation Reporting Standards 2022.Reference Husereau, Drummond, Augustovski, de Bekker-Grob, Briggs and Carswell11

Method

Trial design and participants

This economic evaluation was incorporated into the ASSSIST-2 trial, a pragmatic, two-arm, cluster-randomised controlled trial conducted at the school level. The trial compared the cost-effectiveness of Social Stories plus usual care with usual care alone, for primary-aged children with autism. The protocol for the ASSSIST-2 trial has been published elsewhere.Reference Wright, Teige, Watson, Hodkinson, Marshall and Varley12 In brief, children diagnosed with autism spectrum disorder and aged between 4 and 11 years were recruited from mainstream primary schools in Yorkshire and Humber from November 2018 to May 2021. Parents/guardians and schools were approached to discuss eligibility and provide consent. The detailed inclusion and exclusion criteria can be found in Supplementary File 1, Appendix 1 available at https://doi.org/10.1192/bjo.2024.47. Participating school clusters were randomised in a 1:1 ratio to either the intervention or control condition, using blocked randomisation. Stratification was performed based on school type (special educational needs school or mainstream school) and the number of participating children within the school (five or fewer or more than five participating children). To mitigate against selection bias, all participating children were recruited, and had their baseline assessments completed before school randomisation. Children with autism who were assigned to the intervention arm received Social Stories alongside their usual care and education. Children with autism who were assigned to the control arm received usual care and education only, comprising the routine support typically offered to children with autism by educational and health services. Follow-up assessment was conducted for all children with autism up to 6 months post-randomisation. Refer to Supplementary File 1, Appendix 2 for a flowchart detailing the study.

Intervention

Social Stories was developed by Carol Gray, a specialist teacher, in 1993.Reference Gray and Garand13 It comprises a collection of short stories that usually write the child with autism into a story that includes them in a social situation. They describe, in positive and friendly language, social information about a situation, which the child may be missing or need to know. This can help the child in that situation and can reduce anxiety.Reference William and Wright9 In the current trial, the intervention included training for interventionists and parents/guardians covering autism psychoeducation, design and implementation of Social Stories. Stories were then developed around specific goals that were agreed by teachers, interventionists and parents/guardians to address the child's need for social information. To deliver the intervention, interventionists read the Social Story with the child at least six times over 4 weeks, during school hours, with some level of flexibility to make changes in deliveryReference Wright, Teige, Watson, Hodkinson, Marshall and Varley12 depending on factors such as accessibility for the child and logistical issues at school.

Health outcome measurement

The health outcomes were QALYs measured by the EQ-5D-Y-3L (proxy version).Reference Wille, Badia, Bonsel, Burström, Cavrini and Devlin14 The EQ-5D-Y-3L (proxy version) is a generic preference-based tool assessing health-related quality of life with three severity levels, ranging from 1 (indicating the best health) to 3 (indicating the worst health) over five dimensions/domains (mobility, looking after themselves, doing usual activities, having any pain or discomfort, and feeling worried or sad). This measure allows a proxy person, such as a parent or guardian, to complete it on behalf of children.Reference Wille, Badia, Bonsel, Burström, Cavrini and Devlin14 The EQ-5D-Y-3L has demonstrated reliability and validity as an instrument for assessing health-related quality of life in childrenReference Ravens-Sieberer, Wille, Badia, Bonsel, Burström and Cavrini15 and individuals with a broader category of neurodevelopmental differences, including attention-deficit hyperactivity disorder,Reference Bouwmans, van der Kolk, Oppe, Schawo, Stolk and van Agthoven16 speech/language disorderReference Willems, Joore, Nieman, Severens, Wouters and Hendriks17 and functional disability,Reference Burström, Bartonek, Broström, Sun and Egmar18 suggesting a reasonable basis for EQ-5D-Y-3L's applicability to children and young people with autism.

Responses at the individual level to the EQ-5D-Y-3L proxy were utilised to calculate utility scores, using the Dutch EQ-5D-Y-3L value set.Reference Roudijk, Sajjad, Essers, Lipman, Stalmeier and Finch19 A utility represents a child's health state ‘today’, and ranges from less than 0 (worse than death) to 0 (death) and 1 (full health). The utilities measured at different time points were aggregated through the area under the curve technique, to compute QALYs up to 6 months post-randomisation.Reference Glick, Doshi, Sonnad, Polsky, Glick and Doshi20

Cost measurement

Costs of the Social Stories intervention

Costs associated with the Social Stories intervention encompassed both training costs and the cost related to delivering the intervention. Training costs included preparation time, time spent on delivering training sessions, travel expenses and the material costs during the training. Costs related to delivering the Social Stories intervention included the time spent by professionals to plan and conduct sessions, and the material costs. A bottom-up costing approach was applied,Reference Glick, Doshi, Sonnad, Polsky, Glick and Doshi20 and the staff time costs were estimated by the time spent and the costs per minute, based on salaries. The study team collected all relevant data through the tailored questionnaires.

Cost of service use

Data on service use were gathered through two tailored questionnairesReference Wright, Teige, Watson, Hodkinson, Marshall and Varley12 developed by the research team, based on our previously successful school-based trial, the Investigating Social Competence and Isolation in Children with Autism taking part in LEGO-based Therapy Clubs in School Environments (I-SOCIALISE) trial.Reference Wang, Wright, Bursnall, Cooper, Kingsley and Couteur5 The questionnaire completed by parents/guardians collects information on healthcare services utilisation, including both hospital-based services and those outside a hospital setting, such as community health services, services provided by allied health professionals, mental health services and social services. It also covers school-based services, including those offered by educational psychologists and specialist teacher advisors, as well as parental private expenses and productivity loss. The questionnaire completed by teachers gathers information regarding intervention and support within the school settings, as well as the impact of children's behaviour on school resources. The resource use questionnaires for parents/guardians and teachers are available in Supplementary File 2.

To determine the total cost for each arm, we employed the bottom-up costing approach.Reference Glick, Doshi, Sonnad, Polsky, Glick and Doshi20 The quantity of service use was multiplied by the respective unit costs. Unit costs for healthcare service use were sourced from the 2019/20 National Cost Collection21 and the Unit Costs of Health and Social Care Report 2019.Reference Curtis and Burns22 Medication costs were derived from the Prescription Cost Analysis – England 2019,23 whereas teacher time costs were estimated with data from the Department for Education's 2019 Report.24 Privately paid services were estimated based on market prices, and productivity loss was assessed based on national average wage rates. All costs were expressed in British pounds for the UK financial year 2019–2020. No discount rate was used, as the study time horizon was less than a year (6 months). The list of unit costs is presented in Supplementary File 1, Appendix 3.

Missing data

In the current study, the term ‘complete case’ refers to children with comprehensive utility and cost data at all time points, whereas the ‘base case’ refers to children with missing utility and/or cost data at follow-up, despite having complete baseline assessments. The multiple imputation method via chained equationsReference Rubin25 was employed to impute the identified missing utility and cost data. This imputation process considered trial arm, age, gender, stratification factors (special educational needs status and number of participants), baseline parent-completed Social Responsiveness Scale, Second Edition (SRS-2) score, baseline utility and baseline costs from the societal perspective.

Statistical and economic analysis

From the societal perspective, the primary analysis estimated the incremental cost-effectiveness ratio (ICER), taking into consideration healthcare service costs, education-related service costs, parental out-of-pocket costs (including childcare and private courses) and productivity loss for parents, representing time off work because of their child's autism.

To account for uncertainty around the ICER and imbalanced utility and costs at baseline, a multivariate multilevel model (MMLM) adjusted for clustering and controlled for baseline utility,Reference Manca, Hawkins and Sculpher26 and baseline costs from the societal perspective, age, gender and baseline parent-completed SRS-2 scores were conducted and bootstrapped 5000 times. The application of the MMLM method managed both cost and QALY distributions and addressed their correlation.Reference Gomes, Grieve, Nixon and Edmunds27 Also, the non-parametric cluster-level bootstrap resampling method was used, considering the likely skewness in the distribution of regression residuals.Reference Hoch, Briggs and Willan28 To analyse the imputed data using bootstrapping, we followed the approach proposed by Leurent and colleagues,Reference Leurent, Gomes, Faria, Morris, Grieve and Carpenter29 where bootstrap samples were drawn from each imputed dataset separately, and the estimates were then pooled together. The cost-effectiveness plane was created to visually present 5000 bootstrapped iterations. Furthermore, a cost-effectiveness acceptability curve was drawn to illustrate the likelihood of the intervention being cost-effective across various willingness-to-pay (WTP) thresholds.Reference Fenwick, O'Brien and Briggs30 All analyses were performed with Stata version 16 for Windows (StataCorp, College Station, Texas) based on an intention-to-treat approach.

Sensitivity analysis

To ensure the robustness of the primary analysis results, we performed a series of sensitivity analyses. Initially, a cost–utility analysis (CUA) was performed with complete cases to examine the influence of missing data. Subsequently, another CUA was carried out, focusing on the perspective of the UK National Health Service (NHS) and personal social services (PSS) to evaluate the economic impact solely on the NHS. Following that, a CUA was executed from an NHS/PSS and education perspective to consider the combined economic impact on the NHS and education systems. This was done in recognition of the responsibilities held by both NHS organisations and schools in supporting children with autism, as outlined in the Special Educational Needs and Disability (SEND) Code of Practice 2015.31 Finally, considering that training costs may be viewed as one-time expenses, a CUA was conducted from the societal perspective, excluding training costs. This aimed to evaluate the impact if the intervention were to be provided over an extended period, thereby eliminating the need for continuous training.

Funding, ethics and consent statements

This study was funded by the National Institute for Health and Care Research Health Technology Assessment (HTA) programme (grant number 16/111/91), and the clinical trial registration number is ISRCTN11634810. Approval from the North East – York Research Ethics Committee (approval number 19/NE/0237) was secured. Informed consent in writing was acquired from a parent or person with parental responsibility for each child. The study tool into account to the child's willingness to participate, and those who were not willing were not included. The health economics analysis plan was signed off before analysis, and a copy of the plan is held within the ASSSIST-2 Trial Master File at York Trials Unit, University of York, available for inspection upon reasonable request.

Patient and public involvement

Patient and public involvement is detailed elsewhere.Reference Wright, Teige, Watson, Hodkinson, Marshall and Varley12

Results

Participants

A total of 295 children were screened from 98 schools, and 249 children from 87 schools had baseline assessment and were randomised (129 children from 44 schools were assigned to the Social Stories arm, and 120 children from 43 schools were allocated to the usual care arm). This configuration serves as the base case. Among the participants, 112 children (45.0%) had complete utility and cost data at baseline and the 6-month follow-up, forming the complete case. In the primary analysis, a total of 29.7% of costs or utilities were initially missing and subsequently imputed.

Table 1 shows the baseline characteristics of the participating children. It reveals that three-quarters of children in both the Social Stories and usual care arms were male. This aligns with the demographic distribution of school-age children with autism in the UK.Reference Loomes, Hull and Mandy32 More than 80% in both arms fell within the primary school age range, spanning from 7 to 11 years. Minimal differences were observed in parent SRS-2 and EQ-5D-Y-3L proxy utility scores at the baseline across the arms. In summary, the baseline characteristics demonstrate consistency among arms and different samples (base case and complete case).

Table 1 Key characteristics at baseline by trial arm

SRS-2, Social Responsiveness Scale, Second Edition; SEN, special educational needs.

Health outcome: QALYs

In terms of utility score, both arms showed a slight increase (ranging from 0.01 to 0.04 for utility scores) from baseline to 6-month follow-up, with similar small increments observed in both the base case and complete case samples. Hence, differences between groups in QALYs at 6 months were negligible. Further details are presented in Supplementary File 1, Appendices 4 and 5.

Cost

The cost per session per child for the intervention was estimated to be £15.22, consisting £12.52 of training costs and £2.70 of delivering intervention costs. The key cost driver of training costs was the trainer fee (71.5%), whereas the main contributors to delivery costs were the costs related to the time interventionists spent on preparation and delivery (58.2%) (refer to Supplementary File 1, Appendix 6).

Regarding service costs, the imputed total costs to the society were £1632.4 (95% CI £1160.3–£2104.5) for Social Stories compared with £1713.6 (95% CI £1211.8–£2215.5) for usual care. For the healthcare-related costs, children in the usual care group had higher costs for child and adolescent mental health services-related community-based services, hospital-based services and non-mental health-related medication. The same trend was observed for school-based costs. Children in the Social Stories group incurred less costs in school-based health services (such as educational psychologist and school nurse visits) and in general support from teachers compared with those in usual care. However, higher costs for school-based intervention services (i.e. one-to-one mentoring/individual work and social communication groups) were observed for Social Stories. Finally, it was observed that Social Stories incurred less costs in parental productivity losses, but slightly higher costs in parental private expenses. It is important to note that certain differences in costs may have been influenced by a few cases with high costs. Specifically, the higher school-based intervention service costs and higher private expenses were driven by two cases each. The analysis retained these high-cost cases because they were deemed plausible. However, caution should be exercised in interpreting the cost differences because of the existence of these high-cost cases. Table 2 presents information on the service use costs broken down by perspective, service type, trial arm and before and after imputation, and details of resource use are shown in Supplementary File 1, Appendix 7.

Table 2 Breakdown of the service use costs in six months by trial arm

NHS, National Health Service; PSS, personal social services; CAMHS, Child and Adolescent Mental Health Services.

CUA (primary analysis)

After adjusting for the imbalanced characteristics at baseline, on average, children receiving Social Stories incurred £85.5 (95% CI −£472.6 to £692.8) less costs from the societal perspective (not statistically significant) and maintained similar QALYs (mean incremental difference: 0.001, 95% CI −0.008 to 0.009), compared with those in the usual care group. The 5000 bootstrapped ICER estimates are displayed in Fig. 1(a). It is evident from the illustration that the majority of the simulated estimates fell below the £20 000 per QALY gained threshold line. This implies that if the society is prepared to spend £20 000 for each additional QALY gained, then Social Stories is likely to be the preferred option over usual care, although the incremental cost saving were small and incremental QALYs were similar. The cost-effectiveness acceptability curve in Fig. 1(b) illustrates the likelihood of Social Stories being cost-effective across various WTP thresholds. As shown, if society is willing to spend £20 000 for each QALY gained, the likelihood of Social Stories being a preferred option is 62%, increasing to 63% when the WTP threshold is set at £30 000 per QALY gained.

Fig. 1 Cost-effectiveness plane and cost-effectiveness acceptability curve. QALY, quality-adjusted life-year; WTP, willingness to pay.

Sensitivity analysis

The outcomes of the sensitivity analyses are visualised in Fig. 2 and detailed in Supplementary File 1, Appendix 8. As depicted, the mean incremental costs and QALYs derived from the complete cases closely align with the base case scenario, resulting in a negative cost per QALY gained. Similar results were also observed from the sensitivity analyses conducted from the perspective of the NHS/PSS (scenario 2), the NHS/PSS and education perspective (scenario 3), and the societal perspective excluding training costs (scenario 4). All of the sensitivity analyses demonstrated that Social Stories is dominant, and a good proportion of the bootstrapped estimates fell below the WTP threshold recommended by National Institute for Health and Care Excellence (£20 000 QALY gained), especially when the NHS/PSS and education perspective was adopted (Scenario 3).

Fig. 2 Cost-effectiveness planes of sensitivity analyses. NHS, National Health Service; PSS, personal social services; QALY, quality-adjusted life-year; WTP, willingness to pay.

Discussion

Principal findings

To the best of our knowledge, this study is the first trial-based investigation to evaluate the cost-effectiveness of Carol Gray's Social Stories in children with autism. Our study indicates that, compared with usual care, Social Stories did not lead to an increase in service use costs to the society, and maintained similar QALY improvements. Furthermore, the possibility of Social Stories being cost-effective is <70%, indicating a high degree of uncertainty regarding the cost-effectiveness of the intervention. Sensitivity analyses, incorporating costs measured from different perspectives, yield consistent findings.

Implications of study

The results in Table 2 show a small, but not statistically significant reduction in costs across different perspectives. Such reduction was particularly evident in the education sector, including costs such as school-based health services and general support from teachers and/or teaching assistants in school. Although school-based intervention costs in the Social Stories arm were identified as higher, this appears to be associated with a limited number of high-cost cases in this group rather than indicating a widespread increase in costs across the entire arm. The finding suggests that Social Stories may have the potential to reduce the support needed in schools without affecting the usual school-based services/interventions that children with autism receive. Moreover, since the intervention not only provides social information to children, but also establishes a dialogue between teachers, parents and child, it is likely to improve the adult understanding of children's needs. This, in turn, enables proactive and preventive actions that may lead to less demand for higher level of support and associated costs later on. It is noteworthy that, although our results indicated a small to negligible cost reduction, the prospective savings could hold significance for commissioners when assessing resource utilisation on a larger scale and multiplying the impact by the number of children with autism in the UK. Currently, there are around 98 000 children with autism aged 4–11 years in England,Reference Roman-Urrestarazu, van Kessel, Allison, Matthews, Brayne and Baron-Cohen2,33 and more than 70% of children with autism are educated in mainstream schools.34 This implies a potential cost saving to the NHS and education system of around £9 million for the primary-aged children with autism population as a whole. However, caution is warranted in interpreting this relatively positive finding for two reasons. First, the cost reduction was not statistically significant. Second, a comprehensive long-term analysis is necessary to ensure the sustained effectiveness of Social Stories over an extended period beyond the initial 6-month timeframe, and to verify the persistence of the associated cost savings. If the long-term analysis confirms sustained cost savings, exploring potential joint funding arrangements between the NHS and the education sector could be a viable approach to finance the implementation of Social Stories in the UK. This is because NHS organisations and schools share the responsibilities and financing to support children with autism based on the SEND Code of Practice 2015.31

On the other hand, the disparity in QALY improvements between the two arms was minimal (Supplementary File 1, Appendix 4). Even after considering uncertainty and imbalanced baseline data, the difference in QALYs remains minimal (≦0.001 QALYs), suggesting that Social Stories do not appear to improve the quality of life (measured by EQ-5D-Y-3L proxy) of children with autism. However, using a health-related quality of life instrument more attuned to changes in mental well-being for neurodiverse children and young people could improve future research such as this.

Strengths and limitations

The study is the first to assess the cost-effectiveness of Social Stories, and the data were collected from a fully powered, randomised controlled trial. Such study designs allow more robust estimates to be generated. The adopted multi-perspective approach was another strength. The evaluation accounted for the costs from various perspectives (NHS/PSS, NHS/PSS/education and societal perspectives), making the evaluation results useful to a broad spectrum of stakeholders, including health policy makers, education sectors and parents/guardians of children with autism. Furthermore, the impact of missing data was explored through sensitivity analysis. This approach not only provides reassurance with regards to the robustness of our findings, but also aids policy makers across diverse sectors in making well-informed decisions.

However, the economic evaluation had certain limitations. Chief among them was the issue of children who were unable to receive Social Stories or experienced variance in the frequency of Social Stories delivery within the 4-week time period because of the COVID-19 pandemic. This aspect raises concerns as it has the potential to introduce bias into the study results and underestimate the potential cost saving and QALY improvement. However, since the disruption occurred to both arms, it is expected that the impact on our results is limited. Also, there was a non-negligible amount of missingness in the primary analysis. This amount of missing data may introduce bias and limit the precision of the conclusions. Although the presence of these missing data does introduce additional uncertainty with regards to the substantive conclusion, the apparent insensitivity of the results to different missing-at-random assumptions provides some reassurance. Further, a few high-cost cases were observed, and they may affect the interpretation of certain cost comparison outcomes (see Results). However, these high-cost cases are unlikely to affect the direction of the economic results about Social Stories. This is because these high-cost cases are in the Social Stories arm. Also, although the brevity of EQ-5D-Y was a necessity during the challenging study period (the COVID-19 pandemic), it may not be the optimal instrument for children and young people with autism because of concerns regarding the sensitivity of the EQ-5D to mental health conditions. Although both the EQ-5D and EQ-5D-Y have shown that they can be reliably and validly applied to a broad category of neurodevelopmental differences,Reference Bouwmans, van der Kolk, Oppe, Schawo, Stolk and van Agthoven16Reference Burström, Bartonek, Broström, Sun and Egmar18 future research exploring the utilisation of alternative or supplementary instruments, such as the Pediatric Quality-of-Life Inventory (PedsQL™), is recommended. Finally, this economic evaluation assesses the cost-effectiveness of Social Stories over the short term (6 month), leaving the long-term effects unknown. Although not within the current study's scope, future research would benefit from conducting a model-based economic evaluation. This approach would enable the measurement of long-term cost-effectiveness and the evaluation of the impact on children's productivity as they transition into adulthood.

In conclusion, the current study demonstrates no increase in costs in delivering Social Stories to children with autism in mainstream school settings, and it sustains comparable QALYs. This observation holds true in both primary and sensitivity analyses. The results will be relevant to policy makers, healthcare providers, education sectors and the parents/guardians of children with autism.

Supplementary material

Supplementary material is available online at https://doi.org/10.1192/bjo.2024.47

Data availability

Data supporting the findings can be obtained on reasonable request from the corresponding author, H.I.W. Requests for access to the ASSSIST-2 data will be reviewed on an individual basis by the Chief Investigator.

Acknowledgements

The authors would like to express gratitude to all the participating schools and families for taking part in the trial. We would specifically like to thank Shirley Paul, Natasha Mitchell, David Marshall, Rebecca Joy, Matthew Bailey, Tanya Pawson, Sarah Gardner, James Craggs, Julie Ross, Sharon Bird, Catherine Arthurson and Natalie Day for their contribution to data management and support in overseeing day-to-day operation of the study. We would also like to thank Catarina Teige, Arabella Scantlebury, Leah Attwell, Hayley Carrick, Amelia Taylor, Olivia Taylor, Rachel Hodkinson, Hannah Edwards, Hannah Pearson, Danielle Varley, Rebecca Hargate, Ann Mclaren for their contrition to the report writing. We also extend our appreciation to the following independent members of the Trial Steering Committee: Michael Morton (Chair), James Shearer (Health Economist), Cliona McDowell (Statistician), Holly Judge (Public Involvement Representative), Nikki Totton (Medical Statistician), Sarah Cooper (Sponsor Representative), Alison Thompson (Sponsor Representative) and Sinead Audsley (Sponsor Representative); and the Data Monitoring and Ethics Committee: David Sims (Chair), Lee Shepstone (Medical Statistician) and Linda Davies (Health Economist), for their support and guidance throughout the trial.

Author contributions

H.I.W. and S.P. served as the trial health economists, undertaking data analysis, with H.I.W. taking the lead in manuscript writing. B.W., K.B. and J.B. provided intervention training to education professionals and parents/guardians. C.H. and B.W. acted as chief investigators, conceptualising the study idea and collaborating with S.G. and D.M. on project design. C.W. and L.M. functioned as study statisticians, actively engaged in data analysis and interpretation. K.B., J.B., E.S. and J.W. oversaw the trial, with E.S. directly involved in data acquisition. All authors contributed to data interpretation, and their involvement has been acknowledged through the review and approval of the final manuscript.

Funding

Funding for this study was provided by the National Institute for Health and Care Research (NIHR) Health Technology Assessment (HTA) programme (awarded to B.W. and C.H., grant number 16/111/91). The views and opinions expressed by the authors in this publication are their own and do not necessarily represent the perspectives of the NHS, the NIHR, the HTA programme or the Department of Health and Social Care.

Declaration of interest

B.W. is a Trustee of Biomation CEC and co-author of How to Write Social Stories (Jessica Kingsley Publishers), the reference manual distributed to schools as part of the intervention (any small profits from this book are all fully donated to charity). C.H. is a member of the NIHR Clinical Trials Unit Standing Advisory Committee, HTA Post-Funding Committee teleconference (Programme Oversight Committee members to attend), HTA Funding Committee Policy Group (formerly Clinical Studies Group), HTA Commissioning Committee, HTA – Fast Track Funding Committee and HTA Fast Track Committee – June 2021. All royalties are donated to Martin House Children's Hospice. J.W. has received funding for the following: NIHR Efficacy and Mechanism Evaluation (NIHR132622), NIHR HTA Programme (NIHR131483), NIHR Research for Patient Benefit (RfPB) (NIHR201559), NIHR RfPB (NIHR201176), NIHR HTA Programme (NIHR128625), NIHR Trainees Coordinating Centre (Clinical Lectureship) (ICA-CL-2018-04-ST2-015), NIHR Programme Grant (RP-PG-0618-20001), NIHR Programme Grant (NIHR200607), NIHR Programme Grant (RP-PG-0216-20002), NIHR Programme Grants for Applied Research (PGfAR Programme (RP-PG-0615-20003), NIHR PGfAR Programme (RP-PG-1016-20003), NIHR Post-Doctoral Fellowship (NIHR301807), Cancer Research UK (C11232/A23434), NIHR PGfAR Programme (RP-PG-0109-10020), NIHR RfPB (PB-PG-0816-20029), NIHR RfPB (PB-PG-0317-20047). S.P. declares work on the following projects: Yorkshire Cancer Research ESCAPE: Electronic cigarettes as smoking cessation aid for smokers with Mental Illness pilot and full randomised controlled trial; Medical Research Council Global Challenges Research Fund: Tobacco control capacity programme and Medical Research Council Children Learning about Second-hand Smoke: cluster-randomised controlled trial. The rest of the authors declare no conflicts of interest.

References

American Psychiatric Association. Diagnostic and Statistical Manual of Mental Disorders (5th ed). American Psychiatric Publishing, 2013.Google Scholar
Roman-Urrestarazu, A, van Kessel, R, Allison, C, Matthews, FE, Brayne, C, Baron-Cohen, S. Association of race/ethnicity and social disadvantage with autism prevalence in 7 million school children in England. JAMA Pediatr 2021; 175(6): e210054.CrossRefGoogle Scholar
Buescher, AVS, Cidav, Z, Knapp, M, Mandell, DS. Costs of autism spectrum disorders in the United Kingdom and the United States. JAMA Pediatr 2014; 168(8): 721–8.CrossRefGoogle ScholarPubMed
Lindsay, S, Proulx, M, Thomson, N, Scott, H. Educators’ challenges of including children with autism Spectrum disorder in mainstream classrooms. Int J Disabil Dev Educ 2013; 60(4): 347–62.CrossRefGoogle Scholar
Wang, HI, Wright, BD, Bursnall, M, Cooper, C, Kingsley, E, Couteur, AL, et al. Cost-utility analysis of LEGO based therapy for school children and young people with autism spectrum disorder: results from a randomised controlled trial. BMJ Open 2022; 12(1): e056347.CrossRefGoogle Scholar
Kokina, A, Kern, L. Social story interventions for students with autism spectrum disorders: a meta-analysis. J Autism Dev Disord 2010; 40(7): 812–26.CrossRefGoogle ScholarPubMed
Reynhout, G, Carter, M. Social stories for children with disabilities. J Autism Dev Disord 2006; 36(4): 445–69.CrossRefGoogle ScholarPubMed
Wright, B, Marshall, D, Adamson, J, Ainsworth, H, Ali, S, Allgar, V, et al. Social StoriesTM to alleviate challenging behaviour and social difficulties exhibited by children with autism spectrum disorder in mainstream schools: design of a manualised training toolkit and feasibility study for a cluster randomised controlled trial with nested qualitative and cost-effectiveness components. Health Technol Assess 2016; 20(6): 1258.CrossRefGoogle ScholarPubMed
William, C, Wright, B. A Guide to Writing Social StoriesTM: Step-by-Step Guidelines for Parents and Professionals. Jessica Kingsley Publishers, 2016.Google Scholar
Sansosti, FJ, Powell-Smith, KA, Kincaid, D. A Research Synthesis of Social Story Interventions for Children with Autism Spectrum Disorders. Database of Abstracts of Reviews of Effects (DARE): Quality-Assessed Reviews, Centre for Reviews and Dissemination. 2004.CrossRefGoogle Scholar
Husereau, D, Drummond, M, Augustovski, F, de Bekker-Grob, E, Briggs, AH, Carswell, C, et al. Consolidated health economic evaluation reporting standards 2022 (CHEERS 2022) statement: updated reporting guidance for health economic evaluations. Value Health 2022; 25(1): 39.CrossRefGoogle ScholarPubMed
Wright, B, Teige, C, Watson, J, Hodkinson, R, Marshall, D, Varley, D, et al. Autism Spectrum Social Stories in Schools Trial 2 (ASSSIST2): study protocol for a randomised controlled trial analysing clinical and cost-effectiveness of Social StoriesTM in primary schools. BMC Psychol 2020; 8: 60.CrossRefGoogle ScholarPubMed
Gray, CA, Garand, JD. Social stories: improving responses of students with autism with accurate social information. Focus Autistic Behav 1993; 8(1): 110.CrossRefGoogle Scholar
Wille, N, Badia, X, Bonsel, G, Burström, K, Cavrini, G, Devlin, N, et al. Development of the EQ-5D-Y: a child-friendly version of the EQ-5D. Qual Life Res 2010; 19(6): 875–86.CrossRefGoogle ScholarPubMed
Ravens-Sieberer, U, Wille, N, Badia, X, Bonsel, G, Burström, K, Cavrini, G, et al. Feasibility, reliability, and validity of the EQ-5D-Y: results from a multinational study. Qual Life Res 2010; 19(6): 887–97.CrossRefGoogle ScholarPubMed
Bouwmans, C, van der Kolk, A, Oppe, M, Schawo, S, Stolk, E, van Agthoven, M, et al. Validity and responsiveness of the EQ-5D and the KIDSCREEN-10 in children with ADHD. Eur J Health Econ 2014; 15(9): 967–77.CrossRefGoogle ScholarPubMed
Willems, DCM, Joore, MA, Nieman, FHM, Severens, JL, Wouters, EFM, Hendriks, JJE. Using EQ-5D in children with asthma, rheumatic disorders, diabetes, and speech/language and/or hearing disorders. Int J Technol Assess Health Care 2009; 25(3): 391–9.CrossRefGoogle ScholarPubMed
Burström, K, Bartonek, Å, Broström, EW, Sun, S, Egmar, AC. EQ-5D-Y as a health-related quality of life measure in children and adolescents with functional disability in Sweden: testing feasibility and validity. Acta Paediatr 2014; 103(4): 426–35.CrossRefGoogle ScholarPubMed
Roudijk, B, Sajjad, A, Essers, B, Lipman, S, Stalmeier, P, Finch, AP. A value set for the EQ-5D-Y-3L in The Netherlands. Pharmacoeconomics 2022; 40(Suppl 2): 193203.CrossRefGoogle ScholarPubMed
Glick, HA, Doshi, JA, Sonnad, SS, Polsky, D, Glick, HA, Doshi, JA, et al. Economic Evaluation in Clinical Trials 2nd ed. Oxford University Press, 2014.CrossRefGoogle Scholar
NHS England. 2019/20 National Cost Collection Data Publication. NHS England, 2020 (https://www.england.nhs.uk/publication/2019-20-national-cost-collection-data-publication/).Google Scholar
Curtis, L, Burns, A. Unit Costs of Health and Social Care 2019. Personal Social Services Research Unit, University of Kent, 2019 (https://www.pssru.ac.uk/project-pages/unit-costs/unit-costs-2019/).Google Scholar
NHS Digital. Prescription Cost Analysis - England 2019. Health and Social Care Information Centre, 2020 (https://www.nhsbsa.nhs.uk/statistical-collections/prescription-cost-analysis-england/prescription-cost-analysis-england-2019).Google Scholar
Rubin, DB. Statistical matching using file concatenation with adjusted weights and multiple imputations. J Bus Econ Stat 1986; 4(1): 8794.CrossRefGoogle Scholar
Manca, A, Hawkins, N, Sculpher, MJ. Estimating mean QALYs in trial-based cost-effectiveness analysis: the importance of controlling for baseline utility. Health Econ 2005; 14(5): 487–96.CrossRefGoogle ScholarPubMed
Gomes, M, Grieve, R, Nixon, R, Edmunds, WJ. Statistical methods for cost-effectiveness analyses that use data from cluster randomized trials: a systematic review and checklist for critical appraisal. Med Decis Making 2012; 32(1): 209–20.CrossRefGoogle ScholarPubMed
Hoch, JS, Briggs, AH, Willan, AR. Something old, something new, something borrowed, something blue: a framework for the marriage of health econometrics and cost-effectiveness analysis. Health Econ 2002; 11(5): 415–30.CrossRefGoogle Scholar
Leurent, B, Gomes, M, Faria, R, Morris, S, Grieve, R, Carpenter, JR. Sensitivity analysis for not-at-random missing data in trial-based cost-effectiveness analysis: a tutorial. Pharmacoeconomics 2018; 36(8): 889901.CrossRefGoogle ScholarPubMed
Fenwick, E, O'Brien, BJ, Briggs, A. Cost-effectiveness acceptability curves–facts, fallacies and frequently asked questions. Health Econ 2004; 13(5): 405–15.CrossRefGoogle ScholarPubMed
Loomes, R, Hull, L, Mandy, WPL. What is the male-to-female ratio in autism spectrum disorder? A systematic review and meta-analysis. J Am Acad Child Adolesc Psychiatry 2017; 56(6): 466–74.CrossRefGoogle ScholarPubMed
Department for Education. Special Educational Needs in England, Academic Year 2021/22. Department for Education, 2021 (https://explore-education-statistics.service.gov.uk/find-statistics/special-educational-needs-in-england/2021-22).Google Scholar
Figure 0

Table 1 Key characteristics at baseline by trial arm

Figure 1

Table 2 Breakdown of the service use costs in six months by trial arm

Figure 2

Fig. 1 Cost-effectiveness plane and cost-effectiveness acceptability curve. QALY, quality-adjusted life-year; WTP, willingness to pay.

Figure 3

Fig. 2 Cost-effectiveness planes of sensitivity analyses. NHS, National Health Service; PSS, personal social services; QALY, quality-adjusted life-year; WTP, willingness to pay.

Supplementary material: File

Wang et al. supplementary material 1

Wang et al. supplementary material
Download Wang et al. supplementary material 1(File)
File 670 KB
Supplementary material: File

Wang et al. supplementary material 2

Wang et al. supplementary material
Download Wang et al. supplementary material 2(File)
File 927.1 KB
Submit a response

eLetters

No eLetters have been published for this article.