Mental and psychosocial problems have a broad societal impact on sectors outside health care (1). As the age of onset of mental problems often coincides with the critical period of establishing educational trajectories (Reference Kessler, Amminger, Aguilar-Gaxiola, Alonso, Lee and Ustun2), the economic impact of these problems on the education sector can be particularly substantial (Reference Doran and Kinchin3). Several cost of illness studies have shown that costs in the education sector can constitute a considerable part of the total societal costs of mental and psychosocial problems (4;5). Therefore, the interventions through which children and adolescents with mental and psychosocial problems are targeted are also likely to impact on the resource-use in the education sector. Such costs and benefits that are associated with health interventions but are incurred outside the healthcare sector are known as intersectoral costs and benefits (ICBs). Inclusion of ICBs could be particularly important for economic evaluations conducted from a societal perspective, in order to determine all relevant costs and outcomes associated with an intervention taking place in a broader societal context (Reference Jönsson6).
Among other factors, the lack of validated methodological guidance in the health economic literature has resulted in little emphasis being placed on taking ICBs in the education sector into account in health economics research, in comparison with other sectors, such as informal care or productivity, for which methods and tools are widely available. This is the case, even though in the context of mental health and psychosocial interventions for children and adolescents, education ICBs can be particularly significant (1;3–5;7). Overlooking these ICBs in economic evaluations can have serious implications for the validity of the results, particularly in the context of interventions aimed at youngsters, as their health status has a more significant impact on the resource-use in the education sector, in comparison with older population groups. Furthermore, mental health-related ICBs of younger people in the education sector can even outweigh their resource-use in the healthcare sector (7;8).
Although research into ICBs is limited, the importance of including ICBs in the education sector in health economics research is highlighted in the literature (Reference Evers, Aarts, Alayli-Goebbels, McDaid, Sassi and Merkur9) and in several national pharmacoeconomic guidelines (10;11). Furthermore, Drost et al. (Reference Drost, Paulus, Ruwaard and Evers12) developed an overview of ICBs associated with mental health interventions and distinguished five nonhealthcare sectors, including the education sector, in which ICBs can be incurred. This scheme could be useful for the identification of relevant ICBs, as the first step of the costing process. In addition, several methods for the measurement and valuation of ICBs in the education sector are available (13;14). Nevertheless, current methods focus only on a limited number of education ICBs. Similarly, in the few health economic evaluation studies that did incorporate ICBs in the education sector, the number of items was also limited (8;15). In addition, while the scheme by Drost et al. (Reference Drost, Paulus, Ruwaard and Evers12) is intended for international use, its transferability has not been properly tested in a broader international context. This is of great importance for the ICBs in the education sector in particular, as although valid transferable tools for the identification, measurement, and valuation of ICBs in this sector are needed, they are scarcely available. Furthermore, the proper classification of ICBs in the education sector is still lacking, while this could provide additional value for the development of a consistent harmonized approach for the inclusion of ICBs in health economics research and foster comparability between studies.
To address these knowledge gaps and to make the first steps toward the development of validated tools for including ICBs in the education sector in health economics studies, the objective of this study was threefold. First, this study aimed to further identify education ICBs attributed to mental health interventions. Second, this study aimed to validate the list of the identified ICBs in a broader European context. Third, this study aimed to categorize the identified ICBs as the first step toward the development of a formal classification.
Methods
Context: The PECUNIA Project
This study was conducted alongside the ProgrammE in Costing, resource-use measurement and outcome valuation for Use in multi-sectoral National and International health economic evaluAtions (PECUNIA) project, which aims to tackle the healthcare challenges of an ever-growing and rapidly aging population in the EU by developing new standardized, harmonized, and validated methods and tools for the assessment of costs and outcomes in European healthcare systems (16). The PECUNIA consortium is a network of ten institutions in six European countries (Austria, Germany, Hungary, Spain, the Netherlands, and the United Kingdom [UK]) with relevant complementary methodological expertise. Three mental disorders (depression, schizophrenia, and post-traumatic stress disorder [PTSD]) with high disease, societal, and economic burdens were selected as exemplars for service identification, to address methodological issues and to contribute to major public health challenges with highly needed applied evidence. Methodological choices for this study were made in collaboration with the international health economics and health technology assessment experts within the PECUNIA consortium. For this study, the education sector was defined as encompassing all education services provided to primary- and secondary-level students with (the risk of developing) mental or psychosocial problems.
Identification
Drost et al. (Reference Drost, Paulus, Ruwaard and Evers12) identified fourteen ICBs in the education sector attributed to mental health (preventive) interventions, which were used as a basis for this study. To identify additional ICBs, a search of peer-reviewed and grey literature was conducted in the autumn of 2019. The methodology of the peer-reviewed literature search for this study was similar to the one used by Drost et al. (Reference Drost, Paulus, Ruwaard and Evers12). The search was conducted in PubMed; the search strategy is available on request to the corresponding author. Based on the number of hits (13,137) and the scope of the PECUNIA project, the search was not extended to additional databases and was further limited to the articles pertaining to depression, PTSD, and schizophrenia. To complement the search with potentially relevant national sources and to increase the transferability of the results, grey literature sources (e.g. ministry reports, national [costing] guidelines) were reviewed by co-authors in PECUNIA partner countries. The full texts of peer-reviewed and grey literature sources were reviewed and the studies that did not contain education ICBs were excluded. The literature search generated a pool of ICBs, which were compiled in a list. Similar or duplicated items were clustered under one heading using the classification scheme of Drost et al. (Reference Drost, Paulus, Ruwaard and Evers12) as a reference. Items that referred to healthcare services provided in a school setting (e.g. a school nurse) and country-specific mental health promotion programs were excluded. As a result, an expanded draft list of ICBs in the education sector was developed.
Validation
To validate the transferability of the list of ICBs in the international context, an expert survey was conducted (Supplementary File 1). The aim of the survey was to assess the clarity and relevance of the identified ICBs, and to assess the completeness of the draft list. The survey was conducted with an international group of education experts in the six PECUNIA countries between November 2018 and January 2019. Experts could be either researchers or professionals in the field of education, that is, with knowledge of and/or experience in the education sector. The survey was administered via email in the form of an Excel file. The file contained instructions for completing the survey, survey questions, and the list of ICBs. The ICBs were accompanied by short descriptive definitions, which were developed by the co-authors with the aim of clarifying the meaning of the items to the experts. The email also contained a consent form, in which the experts could give their permission to be acknowledged on the project Web site and in any publications resulting from the survey. The estimated time to complete the survey was 30 minutes. To analyze the responses regarding the clarity and relevance of the ICBs, the answers were summarized to indicate the number of positive, negative, neither positive nor negative (which were regarded as unclear), and missing responses. To assess the completeness of the list, the items suggested by the experts were listed alongside the draft list of ICBs and clustered with previously identified items. The feedback provided by the experts was used to finalize the list of ICBs. It is important to note that due to the exploratory nature of the study, the aim of the expert survey was to gain insight into the experts' opinions about the list of identified ICBs. Hence, the responses were treated as qualitative rather than quantitative data. The survey results were used to finalize the list of ICBs in project team discussions.
To optimize the final list of ICBs via group discussions with relevant health economic experts, it was presented and discussed at multiple international conferences, including the Fourteenth Workshop on Costs and Assessment in Psychiatry (Venice, Italy), the 11th edition of the Lowlands Health Economic Study Group conference (Almen, The Netherlands), the Health Technology Assessment international 2019 Annual Meeting (Cologne, Germany), and the International Health Economic Association 2019 Congress (Basel, Switzerland).
Categorization
Within the PECUNIA project, a conceptual framework for semantic standardization, the PECUNIA care atom, was developed to categorize resource-use items, including the ICBs in the education sector (Reference Simon, Konig, Brodszky, Moscarelli and French17). The draft list was made up of mixed concepts and terms (e.g. services, interventions, outcomes) that needed to be defined and organized in conceptually harmonized clusters with the aim of facilitating the comparability of health systems analyses while taking their complexity into account. The PECUNIA care atom describes the minimum units of analysis of the three phases of the care delivery process: input, throughput, and output. Inputs encompass the resources introduced in a mental healthcare system including services, staff, equipment, consumables, and facilities. Throughputs refer to the resource utilization by consumers and include activities resulting from the delivery of services (e.g. interventions). Outputs refer to the consequences of using care resources such as changes in functioning, morbidity, and mortality associated with mental health and psychosocial problems. For the current study, the final list of ICBs in the education sector was categorized according to this framework.
Results
Identification
The search in PubMed yielded 13,137 hits. After title and abstract screening, 12,941 records were excluded. Full texts of the remaining 196 studies were reviewed and 113 studies were excluded because they did not contain ICBs in the education sector. In addition, thirteen out of ninety-eight reviewed international grey literature sources were included. Ultimately, ICBs in the education sector were extracted from ninety-six peer-reviewed and grey literature sources. The list of peer-reviewed sources is available in Supplementary File 2. Out of the eighty-three selected peer-reviewed sources, twenty focused on mental health in general, thirty-three on depression, five on PTSD, nineteen on schizophrenia, four on depression and anxiety, one on PTSD and depression, and one on schizophrenia and depression. Out of the thirteen grey literature sources, four sources were Web sites of public organizations, three sources were national guidelines, three sources were ministry reports, one source was an unpublished article, one source was a Master's thesis dissertation, and one source was a piece of national legislation. The literature search is outlined in Figure 1.
Fig. 1. Flowchart of the literature search and the development of the list of intersectoral costs and benefits (ICBs); abbreviations: the Netherlands (NL), Austria (AT), Hungary (HU), Germany (DE), Spain (ES), the United Kingdom (UK).
The ICBs retrieved from the literature were clustered in a comprehensive draft list with thirty-eight unique ICBs (Supplementary File 3).
Validation by Experts
Out of thirty-seven experts invited to participate, fourteen experts (38 percent) completed the survey. Each participating country recruited at least one expert. The highest number of responses (n = 4) was obtained in Austria, while the lowest number of experts (n = 1) was recruited in Hungary and Germany. Information on the experts' background and the number of responses per question is presented in Supplementary File 4.
Fourteen experts commented on the clarity of the ICBs. Only a few missing responses were registered. All items were found to be clear by at least eight experts. Twenty-one items were found clear by at least twelve experts. Seventeen items were found unclear by three of four experts. The least clear items were “learning therapy,” “student counseling,” and “low school attainment/productivity/performance.” Four experts mentioned that the difference between “learning therapy” and “special needs diagnostics” was ambiguous, while it was also unclear what “student counseling” entails and why the item “low school attainment/productivity/performance” contained multiple terms under one heading.
Although fourteen experts commented on the relevance of the ICBs, the responses of one expert were excluded as his/her responses to all but one item in this section of the survey were missing. Apart from this, few missing responses were registered. The experts found most items relevant. Thirty-one items were found relevant by twelve or more experts. Over half of the experts found the items “night school” and “attendance officer” the least relevant. The item “night school” was found to be less relevant because it is applicable to adults rather than to younger students. The item “attendance officer” was deemed irrelevant due to cross-country differences. According to the Austrian and German experts, absenteeism is dealt with by regular schools, and there are no special officers responsible for this task.
Eleven experts commented on the completeness of the list. Five experts found the list of ICBs to be complete. Six experts suggested fifteen additional items. The majority of the suggested items had either a degree of overlap with the previously identified ICBs (e.g. “additional lessons” or measures to improve inclusivity in regular classrooms) or could not be allocated to the education sector (e.g. “clinic schools” or “joint work between teaching and health professionals”). Nevertheless, one unique relevant item “support and training services for teachers” was added to the final list based on the suggestions of the experts.
Finalization and Categorization of the List
In addition to “yes”/“no”/“I don't know” answers, the experts provided qualitative comments regarding the clarity and relevance of each item. For example, it was emphasized that “special education services can also be provided in a mainstream environment in the form of additional support” and not only in a special education school. Another expert mentioned that “social reintegration is not necessarily related to the school. Maybe call it school reintegration?” In addition, for some ICBs, the experts mentioned alternative naming used in their country, such as “special need financing” in the UK instead of “student-related financing.” The comments were taken into account when finalizing the list of ICBs. The final list contained twenty-four ICBs, in comparison with thirty-eight ICBs in the draft list. Four items were excluded (“attendance officer,” “night school,” “indirect effect of premature school leave/dropout,” “school-based health promotion interventions”). One item was added based on the experts' comments (“support and training services for teachers”). The overview of the experts' comments and more details about the process of transforming the draft list of ICBs into the final list are presented in Supplementary File 3. Nine items were categorized as inputs, one item was categorized as throughput, and fourteen items were categorized as outputs according to the PECUNIA care atom. The final list of ICBs accompanied by short descriptions is presented in Table 1.
Table 1. Categorization of Intersectoral Costs and Benefits in the Education Sector According to the Care Atom
a ICBs extracted from the scheme by Drost et al. (2013) (Reference Drost, Paulus, Ruwaard and Evers12).
b ICBs extracted from grey literature.
c ICBs added based on the expert survey; inputs—resources introduced in a mental healthcare system (e.g. services, staff, equipment, consumables, and facilities), throughputs—resource utilization and activities necessary for the delivery of services (e.g. interventions), outputs—changes in functioning, morbidity, and mortality associated with mental health and psychosocial problems (e.g. impaired functioning).
d ICBs extracted from peer-reviewed literature.
Discussion
This study aimed to identify relevant ICBs in the education sector, to validate the list of ICBs in a broader European context, and to categorize the ICBs using mental health as a case study. Based on these aims, a comprehensive list of ICBs in the education sector was developed that aims to support and to lay further foundations for the inclusion of important societal costs in health economics research. Although the case of mental health was used to develop the list, it is meant to be generic and applicable to other disease areas. Furthermore, as the list is based on the international literature and has been validated by an international group of experts, it can be used by researchers to select relevant cost items for health economic studies at both national and international levels in a broader European context.
Taking into account the far-reaching effect of health interventions on society is recognized to be a crucial aspect of the overall methodological quality of economic evaluations (Reference Evers, Hiligsmann and Adarkwah18), as well as an important factor in national decision making (Reference Charlton and Rid19). In line with previous research, the current study demonstrates that mental health problems have an (economic) effect on a wide range of sectors in society, including the education sector (3;15). This implies that ICBs in the education sector could constitute a large proportion of total disease costs for certain disease areas, including mental health, and, if relevant to the context of the study, should be included in economic evaluations. In comparison with the scheme of ICBs by Drost et al. (Reference Drost, Paulus, Ruwaard and Evers12), the current overview has been supplemented by the findings from the international grey literature, and its face validity has been assessed by education experts from within a broader European context. Furthermore, the results of this study complement the existing national pharmacoeconomic guidelines that recommend considering ICBs in the education sector in health economics evaluations (10;11).
The categorization of ICBs based on the PECUNIA care atom presents an important step toward the development of a formal ontology of ICBs in the education sector (Reference Nilsson, Baker and Johnston20). Such an ontology will subsequently serve as a basis for the development of standardized measurement and costing tools for the inclusion of these ICBs in health economics research, and ultimately contribute to improving the comparability of health economic analyses. The categorization also demonstrates the heterogeneity of the identified ICBs and implies that for each category of ICBs, different measurement and costing approaches might be needed. While the methods for the measurement and valuation of input and throughput ICBs based on the quantities of resource-use and on the cost-price are relatively straightforward, quantifying ICBs categorized as output can be more complicated. Drost et al. (Reference Drost, Paulus, Ruwaard and Evers14) developed methods for valuing ICBs, in particular those pertaining to changes in the level of economic losses and in the use of services. Nevertheless, for the valuation of the ICBs concerning other effects of mental health problems in the context of the education sector, alternative methods, for example, methods for eliciting stated preferences (Reference Bryan and Dolan21), need to be considered. In addition, some ICBs categorized as output can be measured not only as a cost item but can also be incorporated in the health-related quality of life. This has implications for double counting in economic evaluations and researchers need to be aware of this potential bias.
As mentioned previously, mental and psychosocial problems can affect multiple nonhealthcare sectors besides education. However, the boundaries of the sectors are not always clear. One of the ICBs included in the draft list, “the indirect effects of premature school leaving,” refers to the effect of lower school attainment due to a mental health problem on an individual's success in later life (e.g. lower income, fewer job opportunities). While this item is linked to the education sector, it belongs to the labor sector, as it illustrates the reduced capacity of the individual on the job market. Similar issues were encountered during the literature search. ICBs in other sectors such as health care (e.g. school nurse/physician, education for hospitalized children) and patient and family (e.g. parental stress) were initially extracted from the literature, but subsequently not included in the draft and final lists. There might be several ways to define the intersectoral boundaries. Allocation of ICBs categorized as input (i.e. services or professionals) to a specific sector could be based on the primary aim of the service, while for throughput ICBs (i.e. interventions or activities), allocation could be based on the source of financing. Furthermore, the complex interrelatedness of the sectors indicates the need for a broader approach to conducting health economics research, looking not only at changes in resource-use on the individual level, but also taking into account the environment (e.g. family as a unit of analysis).
Methodological Reflection
This study provides a comprehensive overview of ICBs in the education sector based on the extensive search of peer-reviewed and international grey literature as well as the survey of an international group of experts. While peer-reviewed literature accentuated the relevance of previously identified ICBs, grey literature was of added value, particularly in identifying additional relevant ICBs in the education sector and in developing a more comprehensive list in comparison with the previous overview (Reference Drost, Paulus, Ruwaard and Evers12). Furthermore, the survey with the international experts provided insight into the clarity and relevance of the identified ICBs, and into the transferability of the list in a broader European context. Nevertheless, the current study is largely exploratory in nature and is to be viewed in the context of several limitations. First, the expert survey to validate the list of ICBs was challenging. While recruiting experts via phone proved to be a more effective recruitment strategy in comparison with recruitment via email, the overall response rate was low (38 percent). Second, the list of ICBs was partially based on the peer-reviewed literature pertaining to depression, PTSD, and schizophrenia due to the focus of PECUNIA project. This, while other mental disorders (e.g. autism, attention-deficit hyperactivity disorder) might also be relevant given the study context. Nevertheless, because other sources were not limited to specific mental disorders, it is assumed that the list developed in this study is comprehensive and contains all relevant ICBs in the education sector. Third, although the list of ICBs developed in this study is meant to be generic and applicable to multiple disease areas, it is derived from the mental health literature. Additional literature or qualitative research might be needed to select and prioritize the education ICBs with respect to a specific context. Fourth, the identified ICBs are mainly applicable in the context of primary and secondary education, as these are generally compulsory. Some education ICBs might also be applicable to students with mental problems in higher education, although for the older population consequences in the labor sector might be more relevant.
Implications for Further Research
A broader intersectoral approach to conducting economic evaluations is a new development in the field of health economics, and highlights the need for further work in this direction. First, the impact of the education ICBs on the study outcomes should be further explored. This would potentially demonstrate that ICBs can constitute a large proportion of the total costs and could provide a sound argument for their inclusion in health economics research. Second, to improve the transferability of the results, the ICBs identified in this study need to be further defined and classified to demonstrate their usability, reliability, shareability, portability, and interoperability for being transformed into a formal ontology that can be used across different countries and databases (Reference Gailly and Poels22). One example of such an ontology is DESDE-LTC, the standard taxonomy for description, mapping, and comparison of services for long-term care (Reference Salvador-Carulla, Amaddeo and Gutiérrez-Colosía23), which was selected to classify services identified within the PECUNIA project. DESDE-LTC has been used extensively for assessing the classification of care provision, including specific education services targeting students with mental problems (Reference Romero-López-Alberca, Gutiérrez-Colosía and Salinas-Pérez24). Third, this study forms a basis for the development of standardized measurement and costing tools, which will also be done within the scope of the PECUNIA project. Fourth, while the current list provides a comprehensive overview of ICBs in the education sector, it is also important to investigate which ICBs are the most important to be included in economic evaluations, which could be done by means of best–worst scaling, among other methods.
Policy Implications
The findings indicate that the impact of mental and psychosocial problems on the education sector is indisputable. As current pharmacoeconomic guidelines are shifting toward recommending the adoption of a societal perspective, it is of increasing importance to consider ICBs in the education sector, as these costs might be particularly substantial in the context of certain disease areas and population groups (7;13). Furthermore, by highlighting the far-reaching economic impact of mental health interventions on the education sector and on society in general, this study is in line with the policy approaches that stress the importance of intersectoral collaboration (Reference Walker, Griffin, Asaria, Tsuchiya and Sculpher25), which is particularly relevant given the severe underfunding on mental health (1). Within the scope of the PECUNIA project, this study will ultimately contribute to the development of efficient, evidence-based collaborative care models and intersectoral funding arrangements that are crucial for improving chronic and mental health care in healthcare systems in a broader European context.
Supplementary material
The supplementary material for this article can be found at https://doi.org/10.1017/S0266462320000203.
Acknowledgments
The authors would like to thank Sylvia Heeneman (Maastricht University, Maastricht, The Netherlands), Trudie Schils (Maastricht University, Maastricht, The Netherlands), Gema Gonzalez de Txabarri Miranda (Gobierno Vasco Delegación de Gipuzkoa, Donostia/San Sebastián, Spain), Josefina Isern Pujol (Servei d'Atenció Educativa Inclusiva i Orientació, Barcelona, Spain), Günter Polt (Landesschulrat für Steiermark, Graz, Austria) and nine experts who preferred to remain anonymous for participating in the expert survey. The authors would also like to thank the discussants and others present at the Fourteenth Workshop on Costs and Assessment in Psychiatry (Venice, Italy), the 11th edition of the lolaHESG (lowlands Health Economic Study Group) conference (Almen, The Netherlands), the Health Technology Assessment international (HTAi) 2019 Annual Meeting (Cologne, Germany), and the International Health Economic Association (iHEA) 2019 Congress (Basel, Switzerland) for pleasant discussions after the presentation of the study results.
Source of Funding
This research is part of the ProgrammE in Costing, resource use measurement and outcome valuation for Use in multi-sectoral National and International health economic evaluAtions (PECUNIA) project, which aims to develop a multi-national multi-person multi-sectoral standard for economic evaluations. This project has received funding from the European Union´s Horizon 2020 research and innovation program under grant agreement No 779292.
Conflicts of Interests
None.
Open access
