Hostname: page-component-78c5997874-8bhkd Total loading time: 0 Render date: 2024-11-06T15:24:30.687Z Has data issue: false hasContentIssue false

Identifying patients in need of palliative care: Adaptation of the Necesidades Paliativas CCOMS-ICO© (NECPAL) screening tool for use in Israel

Published online by Cambridge University Press:  26 October 2022

Galia Fisher*
Affiliation:
The Cheryl Spenser Department of Nursing, Faculty of Social Welfare and Health Sciences, University of Haifa, Haifa, Israel Research Department, Shoham Geriatric Medical Center, Pardes Hanna, Israel
Efrat Shadmi
Affiliation:
The Cheryl Spenser Department of Nursing, Faculty of Social Welfare and Health Sciences, University of Haifa, Haifa, Israel
Tammy Porat-Packer
Affiliation:
Research Department, Shoham Geriatric Medical Center, Pardes Hanna, Israel
Anna Zisberg
Affiliation:
The Cheryl Spenser Department of Nursing, Faculty of Social Welfare and Health Sciences, University of Haifa, Haifa, Israel
*
Author for correspondence: Galia Fisher, Research Department, Shoham Geriatric Medical Center, HaNadiv 1, Pardes Hanna 3707101, Israel. Email: [email protected]
Rights & Permissions [Opens in a new window]

Abstract

Objectives

The Necesidades Paliativas CCOMS-ICO© (NECPAL) screening tool was developed to identify patients in need of palliative care and has been used in Israel without formal translation, reliability testing, or validation. Because cultural norms significantly affect subscales such as social vulnerability and health-care delivery, research is needed to comprehensively assess the NECPAL’s components, adapt it, and validate it for an Israeli health-care setting. This study linguistically and culturally translated the NECPAL into Hebrew to examine cultural and contextual acceptability for use in the Israeli geriatric health sector. The newly adapted tool was measured for itemized and scale-level content validity, inter-rater reliability (IRR), and construct validity.

Methods

The NECPAL was back-translated and its content validated by a 5-member expert panel for clarity and relevance, forming the Israeli-NECPAL (I-NECPAL). Six health-care professionals used the I-NECPAL with 25 post-acute geriatric patients to measure IRR. For construct validity, the known-groups method was used, as there is no “gold standard” method for identifying palliative needs for comparison with the NECPAL. The known groups were 2 fictitious cases, predetermined of palliative need. Thirty health-care professionals, blinded to the predetermined palliative status, used the I-NECPAL to determine whether a patient needs a palliative-centered plan of care.

Results

The findings point to acceptable content and construct validity as well as IRR of the I-NECPAL for potential inclusion as a tool for identifying geriatric patients in need of palliative care. Content-validity assessment brought linguistic changes and the exclusion of the frailty parameter from the annex of chronic diseases. The kappa-adjusted scale-level content-validity index indicated a high level of content validity (0.96). IRR indicated a high level of agreement (all parameters with an “excellent–good” agreement level). The sensitivity (0.93), specificity (0.17), positive predictive value (0.53), and negative predictive value (0.71) revealed how heavily the scale weighed upon the surprise question. These metrics are improved when removing the surprise question from the instrument.

Significance of results

Similar to other countries, the Israeli health-care system is regulated by policies that portray the local beliefs and culture as well as evidence-based practice. The decision about when to switch a patient to a palliative-centered plan of care is one such example. It is thus of utmost importance that only locally adapted and vigorously tested screening tools be offered to health-care providers to assist in this decision. The I-NECPAL is the first psychometrically tested palliative needs identification tool for use in the geriatric population in Israel, on both a scale and an itemized level. The results indicate that it can immediately replace the current unvalidated version in use. Further research is needed to determine whether all parts of the scale are relevant for this patient population.

Type
Original Article
Copyright
© The Author(s), 2022. Published by Cambridge University Press.

Introduction

Palliative care is a preventive, anticipatory care management focused on symptom control that maximizes the quality of life in the final stages of the disease. Palliative approaches replace crisis management with prevention; align treatment decisions with patient and family’s physical, psychological, and cultural needs; and aim to improve the quality of life and lessen the use of invasive end-of-life treatment (Amblàs-Novellas et al. Reference Amblàs-Novellas, Murray and Espaulella2016; Buckley Reference Buckley2008; Dharmarajan et al. Reference Dharmarajan, Wang and Lin2017; World Health Organization 2020). Given these benefits, timely attempts to incorporate palliative principles optimize the quality of life for patients with a life-limiting prognosis such as those with advanced cancer, heart failure, or dementia (Amblàs-Novellas et al. Reference Amblàs-Novellas, Murray and Espaulella2016).

Health-care professionals grapple with the decision about when to begin a palliative approach, and general practitioners find it difficult to maintain palliative identification skills (Hui et al. Reference Hui, Paiva and Del Fabbro2019; Shipman et al. Reference Shipman, Gysels and White2008). Historically, palliative care has been limited to acute terminal diseases with a prognosis of weeks or months. The increasing prevalence of chronic diseases has promoted earlier integration of palliative care into care plans for advanced progressive chronic conditions with a limited life prognosis. When to begin such care plans remains unclear (Hui et al. Reference Hui, Paiva and Del Fabbro2019). Patient indicators that signify an approach to the terminal phase include nutritional, functional, and cognitive decline; elevated overall symptom burden; multimorbidity; and use of resources. The dual concepts of decline and severity attempt to capture the changing picture of a chronically ill patient headed toward death and having a growing need for symptomatic control and a waning need for curative treatments. Identifying patients who can benefit from such a palliative approach requires an understanding of the events preceding the terminal phases of various disease trajectories (Amblàs-Novellas et al. Reference Amblàs-Novellas, Murray and Espaulella2016). Screening tools amalgamating various indications of palliative-care needs have arisen to assist in decision-making.

Research on identifying palliative-care needs for disease-specific states, such as heart failure, prognosticate mortality poorly (Allen et al. Reference Allen, Matlock and Shetterly2019). Conversely, it is also difficult to identify when to begin palliative care for patients with multiple chronic conditions (Arbelle et al. Reference Arbelle, Chodick and Goldstein2014). Recent studies of multimorbid patients have found high symptom burden, poor quality of life, and unmet palliative-care needs (Reinke et al. Reference Reinke, Vig and Tartaglione2019). Proposed reasons for these phenomena include the effect of multiple disease states on the individual, their interaction, and the polypharmacy involved in treating them (Calderón-Larrañaga et al. Reference Calderón-Larrañaga, Poblador-Plou and González-Rubio2012; Nunes et al. Reference Nunes, Flores and Mielke2016).

Multimorbidity, defined as 2 or more co-occurring diseases (Fortin et al. Reference Fortin, Soubhi and Bayliss2007), affects 66% of adults over 65 years old, increases with age in both prevalence and severity (Ofori-Asenso et al. Reference Ofori-Asenso, Chin and Curtis2019), and is associated with increased risk of death. For example, evidence indicates that patients with chronic obstructive pulmonary disease are at increased risk of cardiac diseases such as ischemic heart disease, heart failure, and atrial fibrillation. Treatment modalities for 2 or more conditions prove increasingly clinically complicated, at times negating each other (Roversi et al. Reference Roversi, Fabbri and Sin2016).

Studies show that the accuracy of screening tools varies according to patients’ characteristics (Hui et al. Reference Hui, Paiva and Del Fabbro2019). For example, the National Hospice Organization (1996) guidelines identifying the terminal phase of non-oncologic diseases have been widely accepted in the United States despite their questionable predictive accuracy of 6-month mortality (Fox et al. Reference Fox, Landrum-mcniff and Dawson1999). Other tools commonly used include Scotland’s Supportive and Palliative Care Indicators Tool (Highet et al. Reference Highet, Crawford and Murray2014), England’s Gold Standards Framework Prognostic Indicator Guide (GSF-PIG; King et al. Reference King, Thomas and Martin2005), and Spain’s Necesidades Paliativas [Palliative Needs] tool (v.3.1, 2017) (NECPAL) (Gómez-Batiste et al. Reference Gómez-Batiste, Martínez-Muñoz and Blay2013).

The NECPAL tool was developed by the Qualy Observatory/WHO Collaborating Centre for Palliative Care Public Health Programmes of the Catalan Institute of Oncology in Spain to identify advanced terminal patients in need of palliative-care health and social services. It was translated and culturally adapted from the GSF-PIG for population screening throughout the health-care continuum and for use in a family-centered Mediterranean cultural setting (Gómez-Batiste et al. Reference Gómez-Batiste, Martínez-Muñoz and Blay2013). It incorporates a holistic approach, touching on patients’ suffering and social support and functioning by emphasizing palliative triggers such as frailty, multimorbidity, and geriatric syndromes. Studies have shown content and face validity in Spain (Amblàs-Novellas et al. Reference Amblàs-Novellas, Murray and Espaulella2016; Gómez-Batiste et al. Reference Gómez-Batiste, Martínez-Muñoz and Blay2013). The predictive ability of the original NECPAL (v.1.0, 2011) for mortality at 12 and 24 months was found to have high sensitivity (91.3%, 95% CI: 87.2–94.2%) and negative predictive value (NPV) (91.0%, 95% CI: 86.9–94.0). Previously, the NECPAL was used to classify a person’s imminent need for palliative care based on the number of positive parameters (vs. a dichotomous yes/no for palliative needs) (Calsina-Berna et al. Reference Calsina-Berna, Martínez-Muñoz and Bardes Robles2018). Recently, its creators published the NECPAL 4.0 Prognostic (2021) (Gómez-Batiste et al. Reference Gómez-Batiste, Turrillas and Tebé2020; Turrillas et al. Reference Turrillas, Peñafiel and Tebé2021) containing the surprise question (SQ) and 6 of the 13 original parameters, screening for palliative needs by assigning an end-of-life stage based on how many of the 6 prognostic parameters are identified.

The SQ is a single-item assessment tool meant to screen for people approaching the end of life and asks “Would I be surprised if this patient dies within the next (6, 12, or 24) months?” (Lynn et al. Reference Lynn, Schall and Milne2000). Systematic reviews have found wide degrees of predictive accuracy for 12-month mortality, from less than 15% to over 90% (White et  al. Reference White, Kupeli and Vickerstaff2017). The SQ was found to be even less likely to properly discriminate with non-oncologic diagnoses (Downar et al. Reference Downar, Goldman and Pinto2017). In general, the prognostic accuracy of physicians in palliative care among oncologic and non-oncologic diagnoses varies even without specific use of the SQ (White et al. Reference White, Reid and Harris2016). Its use therefore remains relevant only where the patient is well known to the provider and/or as part of a larger multi-item screening tool.

The 13 parameters ask whether there is or has been (1) a request for limitation of therapeutic effort; (2) a provider-identified need for palliative care; (3–5) nutritional, functional, or cognitive decline in the last 6 months unrelated to a reversible process; (6) severe dependence (e.g., bed-bound); (7) multiple recurrent geriatric syndromes (e.g., falls, dysphagia, pressure ulcers, and delirium); (8) multiple persistent symptoms resistant to treatment (e.g., pain, weakness, and digestive issues); (9) severe emotional distress; (10) severe social vulnerability; (11) multimorbidity; (12) increasing use of resources (e.g., at least 3 unplanned admissions in the past 6 months); and (13) disease-specific indicators of severity or progression, titled “The Annex.”

The NECPAL is a complex tool structure comprising multiple subscales of the parameters. For example, functional status can be measured using the Karnofsky Performance Scale (Karnofsky Reference Karnofsky and MacLeod1949), using the Barthel Index (Mahoney and Barthel Reference Mahoney and Barthel1965), or enumerating the declining activities of daily living. Some parameters can be significantly affected by cultural norms (e.g., social vulnerability) or the makeup of health-care delivery (such as readmission rates). The NECPAL therefore requires systematic empirical investigation before adoption into other countries. Its psychometric properties have been assessed after being culturally adapted in Chile (Troncoso et al. Reference Troncoso, Morales-Meyer and Villarroel2021) and Portugal (Santana et al. Reference Santana, Gómez-Batiste and da Silva2020), and it has been used to assess the prevalence of palliative needs among sample populations in other countries (Orzechowski et al. Reference Orzechowski, Galvão and Nunes2019). The original NECPAL has been used here in Israel since 2014 without validation.

The NECPAL’s item-level content validity, inter-rater reliability (IRR), and construct validity via known groups are yet to be studied in the original language or in any translated version to date. No psychometric analysis was found in the assessment of the “Annex” of diseases and their criteria. IRR is of specific importance given the inter-professional nature of palliative care among physicians and nurses when using instruments to identify palliative needs (Myers et al. Reference Myers, Gardiner and Harris2010). The present study undertakes rigorous testing of those additional psychometric properties at both the item and scale levels of the NECPAL that are necessary to determine whether the instrument is appropriate to identify a need for palliative care among geriatric inpatients in Israel.

Methods

This is a two-phase instrument adaptation and validation study.

Phase I: tool formation

Instrument

The NECPAL 2016 v.3.1 was chosen for translation because this version adds more objective frailty measures in an attempt to increase specificity and positive predictive value (PPV; Gómez-Batiste et al. Reference Gómez-Batiste, Amblàs and Costa2017a). It consists of 14 parameters grouped into 4 main categories: I – the SQ; II – demand for palliative care from the patient, caregiver, or provider (parameters 1 and 2); III – general clinical indicators of severity and progression (parameters 3–12); and IV – disease-specific indicators of severity and progression (parameter 13, the “Annex”) (Gómez-Batiste et al. Reference Gómez-Batiste, Amblàs and Costa2017a). The instrument is administered by a health-care provider who either is personally familiar with the patient or has access to the patient’s multidisciplinary (nutritional, functional, cognitive, and/or social) assessments. A patient is considered NECPAL-positive if she/he is positive in category I and one other category. A NECPAL-positive designation implies that a palliative-care approach should be considered. Implications of such an approach include, but are not limited to, discussing advanced directives, do-not-hospitalize orders for patients in end-organ failure, and voluntary refusal of food and fluids and prohibiting curative interventions such as antibiotics and whether to commence life-sustaining treatments such as mechanical ventilation.

Translation and adaptation

The English NECPAL was translated by a native English-speaking registered geriatric nurse with Hebrew proficiency and a native Hebrew-speaking physician with English proficiency working in palliative care. Back-translation was performed by English-speaking professionals with expertise in cross-cultural instrument translation. Original English and back-translated versions were compared for accuracy. Forward- and back-translation, used in similar studies (Lavan et al. Reference Lavan, Gallagher and O’Mahony2018; Santana et al. Reference Santana, Gómez-Batiste and da Silva2020), are part of the Beaton protocol, which is recommended by the World Health Organization (2010).

Content validity

The content-validity assessment package (Lynn Reference Lynn1986) included background information, instructions, a 4-point Likert-scale questionnaire assessing the relevance and clarity of each of the 14 parameters and the disease-specific indicators, and space for qualitative comments. Five expert panelists, working in home care, geriatric hospitals, or academic institutions (2 physicians and 3 nurses with 20 to 40 years of experience in their respective fields), were selected by the researchers for their well-known advocacy and expertise in palliative care, gerontology, or chronic care. Three are certified in palliative care and one in geriatric care. The researchers discussed all first-round comments, adjusted the questions appropriately, and repeated the process a second time. Final feedback was amalgamated into the new, locally adapted, Israeli-NECPAL (I-NECPAL).

Phase II: psychometric testing of the I-NECPAL

Inter-rater reliability

The professionals measuring IRR included 4 physicians and 2 registered nurses working in post-acute geriatric care and certified in either geriatric or palliative care. Together they reviewed 25 medical records of patients at a geriatric medical center in Israel and completed the I-NECPAL tool for each of them. Medical records were randomly chosen for patients discharged in 2019 from rehabilitation, subacute internal medicine, supportive care, frail, or mentally frail units. Power analysis supports a review of 25 records for reaching 80% power, assuming a null hypothesis of kappa to be zero and P ≤ 0.05% (Bujang and Baharum Reference Bujang and Baharum2017; Hong et al. Reference Hong, Choi and Hahn2014; Sim and Wright Reference Sim and Wright2005) in order for a result to be considered meaningful.

Construct validity

Divergent construct validity was assessed via the known-groups method. Two fictitious case studies were written by members of the research team based on experience caring for previous inpatients, one case needing palliative care and one not. Thirty health-care professionals (15 physicians and 15 nurses), almost all certified in at least one specialty (several in more than one: palliative n = 10 [33.3%], geriatric n = 12 [40%], other n = 14 [46.7%]) and working in various settings (community, home care, acute care hospital, post-acute care hospital, and “other”), were asked to complete the I-NECPAL for each case. Power analysis supports a sample size of 30 professionals to reach P ≤ 0.05% and a power of 0.8 to calculate sensitivity and specificity (Faul et al. Reference Faul, Erdfelder and Buchner2009).

Statistical analysis

Content validity

To quantify the content validity of the I-NECPAL, Lynn’s (Reference Lynn1986) suggested stages were used to calculate the content-validity index (CVI) for the individual parameters and for the total scale. Relevance and clarity were rated on a 4-point ordinal scale and grouped (1 and 2 as ratings of agreement; 3 and 4 as disagreement). Each itemized CVI (I-CVI) was adjusted for chance agreement based on the multi-rater kappa (k) statistic adopting the following criteria: Fair = k of 0.40 to 0.59; Good = k of 0.60 to 0.74; and Excellent = k > 0.74 (Cicchetti and Sparrow Reference Cicchetti and Sparrow1981; Fleiss et al. Reference Fleiss, Levin and Paik1981). The overall NECPAL scale-level CVI (S-CVI) was calculated by averaging the kappa values of each I-CVI (Lynn Reference Lynn1986).

Inter-rater reliability

IRR was assessed using Cohen’s kappa, similar to studies of the psychometric properties of other tools for identifying palliative needs (Lavan et al. Reference Lavan, Gallagher and O’Mahony2018). IRR was determined via a percentage agreement similar to an I-CVI but with a multi-rater kappa statistic (Wynd et al. Reference Wynd, Schmidt and Schaefer2003) to analyze the dichotomized data. Criteria for kappa were evaluated using the same guidelines as described above.

Construct validity

A 2 × 2 matrix comparing participants’ decisions on the determination of palliative care using the I-NECPAL was used to measure nonrandom associations and to calculate the sensitivity, specificity, NPV, and PPV (Linsell et al. Reference Linsell, Forbes and Burgess2010). Paired-sample t-tests were performed to test the association between profession (nurse or physician) and certification (palliative or not) on the ability to differentiate significantly the 2 case studies.

Results

Tool formation

Content validity

Table 1 shows the descriptive statistics of the professionals’ ratings for relevance and clarity for each parameter of the translated NECPAL tool in the first and second rounds of expert assessment.

Table 1. Descriptive statistics of content validity of translated tool

Ratings for relevance decreased in the second round for parameters 6 and 13 and neurovascular. Ratings for clarity increased or remained equal in the second round for all parameters except for the SQ, parameters 8 and 9, lung, and neurovascular. The frailty parameter was removed after the second round because of the unanimous view of its redundancy in light of parameters 4 and 6. The final amended tool, the I-NECPAL, was established for testing.

Psychometric testing

Inter-rater reliability

The 25 randomly selected geriatric inpatients (mean age 78.4 ± 8.4 years; male n = 9 [36%], female n = 16 [64%]) were hospitalized in 1 of the 4 inpatient units (subacute n = 6 [24%], rehabilitation n = 6 [24%], supportive care n = 7 [28%], and mentally frail n = 6 [24%]). The analysis of IRR points to good-to-excellent agreement among raters: 10 parameters showed an “excellent” and 4 a “good” level of agreement. Table 2 summarizes the item-level analysis.

Table 2. Inter-rater level of agreement on evaluation of 25 patients using I-NECPAL

Agreement was highest for the SQ among patients in the rehabilitation units (kappa-adjusted CVI = 0.90) and lowest among patients in supportive care (kappa-adjusted CVI = 0.72; Table 3). The groups showed high variability (SD ± 0.16–0.21). Agreement was highest for the remaining parameters among patients in the rehabilitation units (kappa-adjusted CVI = 0.82) and lowest among patients in the dementia care units (kappa-adjusted CVI = 0.72), with lower variability than that for the SQ (SD ± 0.05–0.08).

Table 3. Mean of kappa-adjusted CVI for SQ and I-NECPAL parameters by inpatient department and age group

For the 3 age groups, the highest mean agreement for the SQ was for the oldest (88–93 years) and the lowest was for the youngest (68–71 years). Variability remained high within each age group (SD ± 0.18–0.21). The highest mean agreement of the I-NECPAL tool parameters was for ages 74 to 82 and the lowest was for ages 88 to 93. Variability remained lower for the other parameters than for the SQ.

Construct validity

The results of the 2 × 2 matrix for analyzing the multi-rater (n = 30) dichotomous data are displayed in Table 4. The sensitivity, specificity, PPV, and NPV for the NECPAL were identical to the values calculated for the SQ alone. Use of the NECPAL parameters without the SQ showed decreased sensitivity, increased specificity, and decreased NPV. Use of only those parameters included in the NECPAL 4.0 prognostic gave the best “balance” of values (sensitivity = 0.87, specificity = 0.30, PPV = 0.55, and NPV = 0.69).

Table 4. Measure of sensitivity, specificity, PPV, NPV, and accuracy for each item of the scale in determining known-groups construct validity

a The numeric score was considered accurate for the non-palliative case if it was less than or equal to the intended scoring. The numeric score was considered accurate for the palliative case if it was greater than or equal to the intended scoring.

The paired-sample t-tests portrayed the effect of profession (nurse or physician) and certification (palliative or not) on the ability to differentiate significantly between the 2 case studies using the parameters of the I-NECPAL only, without the SQ. There was a significant difference between the mean score of positive I-NECPAL parameters for the non-palliative case (M = 5.27; t(28) = −2.80, P = 0.01) and the palliative one (M = 8.27; t(28) = −2.80, P = 0.01) when scored by physicians but not when scored by nurses. Furthermore, there was a significant difference between the mean score of positive I-NECPAL parameters for the non-palliative case (M = 5.36; t(27) = −2.82, P = 0.01) and the palliative one (M = 8.27; t(27) = −2.82, P = 0.01) when scored by professionals with a certification in palliative care, regardless of the profession. All differences in means remained significant when only comparing the 6 parameters of the NECPAL 4.0 prognostic (parameters 2, 3, 4, 11, 12, and 13).

Discussion

This is the first study to validate the NECPAL for use in Israel and the first to assess the item-level content validity, IRR, and construct validity of the NECPAL in any language. The results of the psychometric testing of the adapted tool, the I-NECPAL, point to acceptable content and construct validity and the IRR of the I-NECPAL for potential inclusion as a tool for identifying geriatric patients in need of palliative care.

The construct-validity phase revealed how heavily the scale weighs on the SQ, sharing the same PPV, NPV, sensitivity, and specificity values. The geriatric post-acute setting is challenging. Although the cases, presented as inpatient geriatric cases, differ in their palliative needs, most of the practitioners would not have been surprised if either had died within the year. Because the NECPAL requires a positive answer (I would be surprised if this patient died within the next 12 months) for a patient to be considered NECPAL-positive, the currently accepted scoring system may be irrelevant for this population. This is further supported by the results of the IRR analysis, which were highest for the SQ among older patients as well as those in rehabilitation and dementia departments, where the illness trajectory is more uniform. On the other hand, agreement for the youngest group and for those patients in subacute and supportive-care departments, where the illness trajectory is sometimes less clear, was lowest.

This study additionally analyzed the I-NECPAL without the SQ. When considering the objective parameters alone, agreement is excellent for patients in supportive care, rehabilitation, and subacute departments but only good for those in dementia units. When analyzed by age, agreement of the parameters alone was excellent for younger patients but only good in the oldest group. Use of the scale’s objective parameters is less uniformly understood for the longer illness trajectory, that is, dementia but also for the oldest group of patients.

Because this study also measured item-level properties, one can draw inferences about the newer shortened version of the NECPAL 4.0 prognostic by assessing the measures for the relevant items, as this tool shares the SQ as well as parameters 2 through 4 and 11 through 13. The content-validity assessment scored these parameters all as “very relevant/very clear” (scores 3.8–4.0). The IRR was excellent for each of these, except for parameter 2, for which it was “good.” Finally, the measures for sensitivity, PPV, and NPV all increased when focusing on these parameters alone, whereas specificity remained the same. Practically, this means that based on the provided case studies, the prognostic sensitivity of the NECPAL 4.0 portrays the tool’s ability to detect 87% of patients truly in true of palliative care and 30% of patients who do not yet need such care. Of note is that the divergent case studies presented scored 3 and 4 on the relevant NECPAL 4.0 parameters, placing them in the same prognostic stage II with a median of 17.2 months of survival. This may attest only to the ability of palliative certified professionals to significantly differentiate between the cases. Clearly, further study of the implementation of the I-NECPAL in geriatric settings in Israel is warranted.

Limitations of the current study include the type of case studies written for known-groups construct validity. It is possible that these studies were too similar in complexity to non-geriatricians for divergence to be clearly seen using the I-NECPAL, as is evidenced by the lower PPV and NPV. Still, because the tool is meant to also work for subtle cases, this may also be a factor in the weight the SQ holds in the assessment. In addition, there may not have been enough information provided for each case, but sometimes only a little bit of information is available to a health-care professional at the time. Finally, while the IRR phase used a small but homogenous sample of practitioners regarding practice setting, the construct-validity phase used a small and largely heterogenous sample regarding practice setting.

This study’s clinical implications include providing a reliable and validated identification tool for Israeli health-care practitioners in making decisions at pivotal moments along a patient’s life course. Such moments can span weeks or months, are thus difficult to identify, and together form the shift from an all-curative-centered plan of care to one that focuses on alleviating symptoms. These decisions include the culturally sensitive issue of when it is appropriate to discuss advanced directives and the more personally meaningful issue of what defines quality as we approach the end of life. National Health System policies addressing the end-of-life care vary from country to country, reflecting local beliefs and culture as well as evidence-based practice. Practitioners now have not only a validated and reliable I-NECPAL to help with these decisions but an initial evidence base to soundly consider whether to use the SQ based on their own palliative expertise.

Acknowledgments

The authors thank those who participated in this study, which was approved by the Shoham Medical Center institutional review board (IRB) (no. 1.20) and the University of Haifa Faculty of Social Welfare and Health Sciences IRB (no. 2568). Written permission was obtained from Xavier Gómez-Batiste Alentorn, MD, PhD, to translate and test the psychometric properties of the NECPAL into Hebrew.

Funding

This research received no specific grant from any funding agency, commercial, or not-for-profit sectors.

Conflicts of interest

None declared.

References

Allen, LA, Matlock, DD, Shetterly, SM, et al. (2019) Use of risk models to predict death in the next year among individual ambulatory patients with heart failure. Journal of the American Medical Association Cardiology 2(4), 435441. doi:10.1001/jamacardio.2016.5036Google Scholar
Amblàs-Novellas, J, Murray, SA, Espaulella, J, et al. (2016) Identifying patients with advanced chronic conditions for a progressive palliative care approach: A cross-sectional study of prognostic indicators related to end-of-life trajectories. BMJ Open 6, 110. doi:10.1136/bmjopen-2016-012340CrossRefGoogle ScholarPubMed
Arbelle, JE, Chodick, G, Goldstein, A, et al. (2014) Multiple chronic disorders: Health care system’s modern challenge in the Maccabi Health Care System. Israel Journal of Health Policy Research 3(1), 18. doi:10.1186/2045-4015-3-29CrossRefGoogle ScholarPubMed
Buckley, J (2008) Palliative Care: An Integrated Approach. Chichester: John Wiley & Sons Ltd.CrossRefGoogle Scholar
Bujang, MA and Baharum, N (2017) Guidelines of the minimum sample size requirements for Cohen’s Kappa. Epidemiology Biostatistics and Public Health 14(2), -1–e12267-10. doi:10.2427/12267Google Scholar
Calderón-Larrañaga, A, Poblador-Plou, B, González-Rubio, F, et al. (2012) Multimorbidity, polypharmacy, referrals, and adverse drug events: Are we doing things well? British Journal of General Practice 62(605), 821826. doi:10.3399/bjgp12X659295CrossRefGoogle ScholarPubMed
Calsina-Berna, A, Martínez-Muñoz, M, Bardes Robles, I, et al. (2018) Intrahospital mortality and survival of patients with advanced chronic illnesses in a tertiary hospital identified with the NECPAL CCOMS-ICO tool. Journal of Palliative Medicine 21(5), 665673. doi:10.1089/jpm.2017.0339CrossRefGoogle Scholar
Cicchetti, DV and Sparrow, SA (1981) Developing criteria for establishing interrater reliability of specific items: Applications to assessment of adaptive behavior. American Journal of Mental Deficiency 86(2), 127137.Google ScholarPubMed
Dharmarajan, K, Wang, Y, Lin, Z, et al. (2017) Association of changing hospital readmission rates with mortality rates after hospital discharge. Journal of the American Medical Association 318(3), 270278. doi:10.1001/jama.2017.8444CrossRefGoogle ScholarPubMed
Downar, J, Goldman, R, Pinto, R, et al. (2017) The “surprise question” for predicting death in seriously ill patients: A systematic review and meta-analysis. Canadian Medical Association Journal 189(13), E484E493. doi:10.1503/cmaj.160775CrossRefGoogle Scholar
Faul, F, Erdfelder, E, Buchner, A, et al. (2009) Statistical power analyses using G*Power 3.1: Tests for correlation and regression analyses. Behavior Research Methods 41(4), 11491160. doi:10.3758/BRM.41.4.1149CrossRefGoogle ScholarPubMed
Fleiss, JL, Levin, B and Paik, MC (1981) The measurement of interrater agreement. Statistical Methods for Rates and Proportions 2, 212236.Google Scholar
Fortin, M, Soubhi, H, Bayliss, E, et al. (2007) Mutimorbidity’s many challenges. British Medical Journal 334(7602), 10161017. doi:10.1136/bmj.39201.463819.2CCrossRefGoogle ScholarPubMed
Fox, E, Landrum-mcniff, K, Dawson, NV, et al. (1999) Evaluation of prognostic criteria for determining hospice eligibility. Journal of the American Medical Association 282(17), 16381645. doi:10.1001/jama.282.17.1638CrossRefGoogle ScholarPubMed
Gómez-Batiste, X, Amblàs, J, Costa, X, et al. (2017a) Recommendations for the Comprehensive and Integrated Care of Persons With Advanced Chronic Conditions and Life-Limited Prognosis in Health and Social Services: NECPAL CCOMS-ICO© 3.1 (2017). Catalan Institute of Oncology. https://ico.gencat.cat/ca/professionals/serveis_i_programes/observatori_qualy/eines_de_suport/eines/instrument_i_programa/.Google Scholar
Gómez-Batiste, X, Martínez-Muñoz, M, Blay, C, et al. (2013) Identificacion de personas con enfermedades crónicas avanzadas y necesidad de atencion paliativa en servicios sanitarios y sociales: Elaboración del de atencio instrumento NECPAL CCOMS-ICO. Medicina Clínica (Barcelona) 140(6), 241245. doi:10.1016/j.medcli.2012.06.027CrossRefGoogle Scholar
Gómez-Batiste, X, Turrillas, P, Tebé, C, et al. (2020) NECPAL tool prognostication in advanced chronic illness: A rapid review and expert consensus. BMJ Supportive & Palliative Care, 111. doi:10.1136/bmjspcare-2019-002126Google ScholarPubMed
Highet, G, Crawford, D, Murray, SA, et al. (2014) Development and evaluation of the supportive and palliative care indicators tool (SPICT): A mixed-methods study. BMJ Supportive & Palliative Care 4(3), 285290. doi:10.1136/bmjspcare-2013-000488CrossRefGoogle ScholarPubMed
Hong, H, Choi, Y, Hahn, S, et al. (2014) Nomogram for sample size calculation on a straightforward basis for the kappa statistic. Annals of Epidemiology 24(9), 673680. doi:10.1016/j.annepidem.2014.06.097CrossRefGoogle ScholarPubMed
Hui, D, Paiva, CE, Del Fabbro, EG, et al. (2019) Prognostication in advanced cancer: Update and directions for future research. Supportive Care in Cancer 27(6), 19731984. doi:10.1007/s00520-019-04727-yCrossRefGoogle ScholarPubMed
Karnofsky, DA (1949) The clinical evaluation of chemotherapeutic agents in cancer. In MacLeod, CM (ed), Evaluation of Chemotherapeutic Agents. New York: Columbia University Press, 191205.Google Scholar
King, N, Thomas, K, Martin, N, et al. (2005) “Now nobody falls through the net”: Practitioners’ perspectives on the Gold Standards Framework for community palliative care. Palliative Medicine 19(8), 619627. doi:10.1191/0269216305pm1084oaCrossRefGoogle ScholarPubMed
Lavan, AH, Gallagher, P and O’Mahony, D (2018) Inter-rater reliability of STOPPFrail [Screening Tool of Older Persons Prescriptions in Frail adults with limited life expectancy] criteria amongst 12 physicians. European Journal of Clinical Pharmacology 74, 331338. doi:10.1007/s00228-017-2376-2CrossRefGoogle ScholarPubMed
Linsell, L, Forbes, LJL, Burgess, C, et al. (2010) Validation of a measurement tool to assess awareness of breast cancer. European Journal of Cancer 46(8), 13741381. doi:10.1016/j.ejca.2010.02.034CrossRefGoogle ScholarPubMed
Lynn, MR (1986) Determination and quantification of content validity. Nursing Research 35(6), 382386. doi:10.1097/00006199-198611000-00017CrossRefGoogle ScholarPubMed
Lynn, J, Schall, MW, Milne, C, et al. (2000) Quality improvements in end of life care: Insights from two collaboratives. The Joint Commission Journal on Quality Improvement 26(5), 254267. doi:10.1016/S1070-3241(00)26020-3CrossRefGoogle ScholarPubMed
Mahoney, FI and Barthel, DW (1965) Functional evaluation: The Barthel index. Maryland State Medical Journal 14, 6165.Google ScholarPubMed
Myers, J, Gardiner, K, Harris, K, et al. (2010) Evaluating correlation and interrater reliability for four performance scales in the palliative care setting. Journal of Pain and Symptom Management 39(2), 250258. doi:10.1016/j.jpainsymman.2009.06.013CrossRefGoogle ScholarPubMed
National Hospice Organization (1996) Medical guidelines for determining prognosis in selected non-cancer diseases. The Hospice Journal 11(2), 4763.10.1080/0742-969X.1996.11882820CrossRefGoogle Scholar
Nunes, BP, Flores, TR, Mielke, GI, et al. (2016) Multimorbidity and mortality in older adults: A systematic review and meta-analysis. Archives of Gerontology and Geriatrics 67, 130138. doi:10.1016/j.archger.2016.07.008CrossRefGoogle ScholarPubMed
Ofori-Asenso, R, Chin, KL, Curtis, AJ, et al. (2019) Recent patterns of multimorbidity among older adults in high-income countries. Population Health Management 22(2), 127137. doi:10.1089/pop.2018.0069CrossRefGoogle ScholarPubMed
Orzechowski, R, Galvão, AL, Nunes, TDS, et al. (2019) Palliative care need in patients with advanced heart failure hospitalized in a tertiary hospital. Revista da Escola de Enfermagem da USP 53, . doi:10.1590/S1980-220X2018015403413CrossRefGoogle ScholarPubMed
Reinke, LF, Vig, EK, Tartaglione, EV, et al. (2019) Symptom burden and palliative care needs among high-risk veterans with multimorbidity. Journal of Pain and Symptom Management 57(5), 880889. doi:10.1016/j.jpainsymman.2019.02.011CrossRefGoogle ScholarPubMed
Roversi, S, Fabbri, LM, Sin, DD, et al. (2016) Chronic obstructive pulmonary disease and cardiac diseases an urgent need for integrated care. American Journal of Respiratory and Critical Care Medicine 194(11), 13191336. doi:10.1164/rccm.201604-0690SOCrossRefGoogle ScholarPubMed
Santana, MTEA, Gómez-Batiste, X, da Silva, LMG, et al. (2020) Cross-cultural adaptation and semantic validation of an instrument to identify palliative requirements in Portuguese. Einstein (Sao Paulo, Brazil) 18, . doi:10.31744/einstein_journal/2020AO5539Google ScholarPubMed
Shipman, C, Gysels, M, White, P, et al. (2008) Improving generalist end of life care: National consultation with practitioners, commissioners, academics, and service. BMJ 337(a1720), 18. doi:10.1136/bmj.a1720CrossRefGoogle ScholarPubMed
Sim, J and Wright, CC (2005) The Kappa statistic in reliability studies: Use, interpretation, and sample size requirements. Physical Therapy 85(3), 257268. doi:10.1093/ptj/85.3.257CrossRefGoogle ScholarPubMed
Troncoso, J, Morales-Meyer, T, Villarroel, L, et al. (2021) Adaptation and validation in Chile of the patient identification instrument needing palliative care: NECPAL-CCOMS-ICO 3.1©. Atencion Primaria 53(4), 101994. doi:10.1016/j.aprim.2021.101994CrossRefGoogle ScholarPubMed
Turrillas, P, Peñafiel, J, Tebé, C, et al. (2021) NECPAL prognostic tool: A palliative medicine retrospective cohort study. BMJ Supportive & Palliative Care, 110. doi:10.1136/bmjspcare-2020-002567Google Scholar
White, N, Kupeli, N, Vickerstaff, V, et al. (2017) How accurate is the “surprise question” at identifying patients at the end of life? A systematic review and meta-analysis. BMC Medicine 15(139), 114. doi:10.1186/s12916-017-0907-4CrossRefGoogle ScholarPubMed
White, N, Reid, F, Harris, A, et al. (2016) A systematic review of predictions of survival in palliative care: How accurate are clinicians and who are the experts? PLoS One 11(8), 120. doi:10.1371/journal.pone.0161407CrossRefGoogle ScholarPubMed
World Health Organization (2010) Process of Translation and Adaptation of Instruments. Geneva, Switzerland: World Health Organization.Google Scholar
World Health Organization (2020) Palliative care. https://www.who.int/news-room/fact-sheets/detail/palliative-care (accessed 8 November 2019).Google Scholar
Wynd, CA, Schmidt, B and Schaefer, MA (2003) Two quantitative approaches for estimating content validity. Western Journal of Nursing Research 25(5), 508518. doi:10.1177/0193945903252998CrossRefGoogle ScholarPubMed
Figure 0

Table 1. Descriptive statistics of content validity of translated tool

Figure 1

Table 2. Inter-rater level of agreement on evaluation of 25 patients using I-NECPAL

Figure 2

Table 3. Mean of kappa-adjusted CVI for SQ and I-NECPAL parameters by inpatient department and age group

Figure 3

Table 4. Measure of sensitivity, specificity, PPV, NPV, and accuracy for each item of the scale in determining known-groups construct validity