Background
Long-term care for older people can be described as support and care activities undertaken to ensure people with loss of function and capacity are able to maintain their wellbeing (Organisation for Economic Co-operation and Development (OECD) 2005). This can encompass care provided within the individual's own home, or care provided in institutional settings (nursing or aged-care homes). The long-term care sectors of many countries are going through processes of transformation, partly in response to increasing calls to change from a ‘one-size-fits-all’ approach to a flexible system with care individualised to the needs and preferences of the consumer (Alakeson Reference Alakeson2010; Glendinning et al. Reference Glendinning, Challis, Fernández, Jacobs, Jones, Knapp, Manthorpe, Moran, Netten, Stevens and Wilberforce2008). In discussing the consumers of long-term care, it should be acknowledged that this encompasses residents themselves, first and foremost, but also can be considered to include the informal family member carers of residents. Family members are often highly involved in providing informal care and decision making for health and social care for older people in residential care, especially where the resident has severe cognitive impairment or where decision making is formally passed on to them (World Health Organization 2012). Cognitive impairment is highly prevalent among long-term care users (Bjork et al. Reference Bjork, Juthberg, Lindkvist, Wimo, Sandman, Winblad and Edvardsson2016; Ribbe et al. Reference Ribbe, Ljunggren, Steel, Topinkova, Hawes, Ikegami, Henrard and Jónnson1997). Currently over half of residents in Australian long-term care facilities have a diagnosis of dementia (Australian Institute of Health and Welfare 2016).
Long-term care costs remain a significant source of public expenditure, varying from 0.2 to 3 per cent of Gross Domestic Product in OECD participating countries, with institutionally based care (such as nursing or residential homes) accounting for the greatest proportion of costs (OECD 2005). As with all public monies, there is a need to ensure that this is a worthwhile use of these funds to ensure ‘value for money’ for the public (Drummond et al. Reference Drummond, Sculpher, Torrance, O'Brien and Stoddart2005). In the health sector, such considerations of worthwhileness have traditionally included the perspectives of the general population, since in many countries health systems are financed by significant funding from governments using tax revenue ultimately generated from its citizens (Brazier et al. Reference Brazier, Ratcliffe, Salomon and Tsuchiya2017). Increasingly for the health and social care sectors, however, the views of care recipients and in some cases informal carers are being considered important to this process due to the unique perspective and increased familiarity they have with the benefits of these services, in addition to the fact they are being asked increasingly to co-fund their care in some countries (Brazier et al. Reference Brazier, Ratcliffe, Salomon and Tsuchiya2017; Coast Reference Coast2004). Measuring the quality of care provided in these services is an important component to the promotion of efficiency and ensuring the appropriate use of public expenditures (Castle and Ferguson Reference Castle and Ferguson2010). Although many definitions of quality of care exist, salient features of the quality of care in this context may include the organisational culture, medical/clinical care, physical, cognitive and emotional functioning, psycho-social support, and fulfilment of the resident's basic rights including dignity, autonomy and privacy (Castle and Ferguson Reference Castle and Ferguson2010; Kane Reference Kane2001; Killett et al. Reference Killett, Burns, Kelly, Brooker, Bowes, La Fontaine, Latham, Wilson and O'Neill2016).
Donabedian (Reference Donabedian1985) proposed a theoretical framework for indicators of quality of care including structures (i.e. organisational characteristics associated with provision of care), processes (i.e. tasks undertaken with or for the resident) and outcomes (i.e. the desired states the care is aiming to achieve). Traditionally, the concepts of quality of care adopted in long-term care settings have not strongly incorporated the resident's own voice and have tended to focus predominantly on the measurement of organisational and clinical processes as quality indicators (Castle and Ferguson Reference Castle and Ferguson2010; Nakrem et al. Reference Nakrem, Vinsnes, Harkless, Paulsen and Seim2009). Concerns have been raised that such indicators produce a high level of focus on paper compliance rather than activities which are supporting the resident's wellbeing, and that structural and clinical care focused measures are not well correlated with improvement in quality of life for residents (Kane Reference Kane2001; Rahman and Applebaum Reference Rahman and Applebaum2009).
With the recent introduction of consumer-directed care in the Australian aged-care sector and other similar policy initiatives internationally, the perspectives of consumers of long-term care services have become increasingly important to concepts of care quality (Castle and Ferguson Reference Castle and Ferguson2010). Previous studies have indicated aspects of care provision in long-term care that are of importance to residents and their family members, including characteristics of the care provided and the staff providing the care, support for the personhood of the individual, the physical and social environment, and level of autonomy for the individual (Bergland and Kirkevold Reference Bergland and Kirkevold2006; Custers et al. Reference Custers, Westerhof, Kuin, Gerritsen and Riksen-Walraven2012; Kane et al. Reference Kane, Caplan, Urv-Wong, Freeman, Aroskar and Finch1997; Schenk et al. Reference Schenk, Meyer, Behr, Kuhlmey and Holzhausen2013).
In order for long-term care facilities to better meet the needs and expectations of residents and their family member carers, a set of quality indicators are needed to evaluate the quality of care from their perspective. Previous work has indicated that instruments specifically designed for this purpose, for application with consumers in long-term care facilities, may perform better than generic instruments, such as those designed for measuring generic health-related quality of life (Forder and Caiels Reference Forder and Caiels2011).
The Consumer Choice Index – Six Dimension (CCI-6D) instrument was designed to fill this gap. The CCI-6D represents the first instrument in Australia and internationally developed from its inception with consumers (including both residents and family member carers) to evaluate the quality of care received in long-term care facilities from a consumer perspective. The aim of this paper is to describe the process of development and assessment of the validity of the CCI-6D instrument descriptive system.
Methods
Development of the CCI-6D
A five-stage process was undertaken to develop the CCI-6D instrument following the stages described by Stevens and Palfreyman (Reference Stevens and Palfreyman2012). An initial literature review highlighted that the vast majority of conceptual frameworks had been developed without any co-production or co-development involving people with dementia. Hence we firstly undertook a qualitative study with people with dementia, on the key determinants of quality in long-term care facilities, to identify potential candidate dimensions (stage 1) (see Milte et al. Reference Milte, Shulver, Killington, Bradley, Ratcliffe and Crotty2016). The first four stages of development of the CCI-6D instrument are described in detail in elsewhere, but in brief we undertook a data collection (stage 2) via one-to-one interviews with residents supplemented by a series of focus groups undertaken with family members of the people with more moderate and severe cognitive impairment, using a semi-structured approach (stage 3) with questions focusing around the characteristics in a care home that would make it a place people either wanted to reside in or did not. The data were analysed (stage 4) in batches as the data collection progressed until ‘data saturation’ (i.e. the point where no new themes within the data were identified) was achieved. A number of key dimensions were identified in the analysis, including ‘choice, freedom and self-determination’, ‘access to meaningful activities’, ‘feeling useful and valued’, ‘respect of possessions and personal space’, ‘individualised care’, ‘supportive health care and medical management’, ‘home-like spaces’ and ‘supportive safety techniques’ (Milte et al. Reference Milte, Shulver, Killington, Bradley, Ratcliffe and Crotty2016). These dimensions were also found to align with the concepts of individualised care (Chappell, Reid and Gish Reference Chappell, Reid and Gish2007), personhood for dementia care (Kitwood and Bredin Reference Kitwood and Bredin1992), and flexibility and autonomy in resident daily life (Boyle Reference Boyle2004). As the instrument needed to be parsimonious for ease of use whilst also encapsulating the key dimensions of importance, these key concepts were distilled down by the research team to those that were most often and consistently raised, to generate six candidate dimensions for inclusion within the CCI-6D instrument. These six dimensions were then presented to our project advisory group (N = 10, comprising consumer representatives from the Alzheimer's Australia Consumer Dementia Research Network (who were informal care-givers), clinicians, health services researchers and representatives of long-term care providers) for confirmation of consumer-friendly wording and face validity.
The fifth and final stage involved generating the descriptive system. The CCI-6D instrument comprises six main dimensions reflecting quality of care from the consumer perspective: (a) time spent attending to individual needs (Care Time), (b) home-like and non-institutional shared spaces (Shared Spaces), (c) home-like and non-institutional own rooms (Own Room), (d) access to outside and garden space (Outside and Gardens), (e) access to participation in meaningful activities individualised for the person (Meaningful Activities), (f) flexibility in the time that care activities were undertaken (Care Flexibility). These dimensions were operationalised as the six questions of the CCI-6D instrument, each with three different levels of measurement ranging from low to high levels of performance. In addition, the research team plans to develop a preference-weighted scoring algorithm, to complement the descriptive system of the CCI-6D as a second stage of this project, to enable incorporation of the instrument into an economic evaluation framework.
Approach for assessment of validity
For a newly developed instrument such as this, it is important to evaluate whether the instrument measures what was originally intended (i.e. in the case of the CCI-6D the quality of care in a long-term care facility). There are a number of different types of validity, including content, construct and criterion-related validity (Bowling Reference Bowling, Bowling and Ebrahim2005). Content validity refers to the extent to which the content of the instrument appears to be measuring the full scope of the characteristics it is intended to measure. This was provided initially through our qualitative interview process with older people and family member carers, and assessed through consultation with our project advisory group later. Criterion validity refers to the evaluation of a new instrument against an existing gold-standard, however, in the case of the quality of care received in long-term care facilities there is no such recognised instrument. Accordingly, our assessment of validity for the CCI-6D follows that taken by Coast et al. (Reference Coast, Peters, Natarajan, Sproston and Flynn2008) and Malley et al. (Reference Malley, Towers, Netten, Brazier, Forder and Flynn2012), and focuses on the construct validity of each of the dimensions, i.e. whether the six dimensions of the new instrument exhibited expected relationships with other variables and concepts. Previous studies have noted that for preference-weighted instruments the requirements for assessing validity differ from the method for psychometric assessment of instruments more generally (Brazier, Deverill and Green Reference Brazier, Deverill and Green1999; Coast et al. Reference Coast, Peters, Natarajan, Sproston and Flynn2008; Malley et al. Reference Malley, Towers, Netten, Brazier, Forder and Flynn2012). That is, it requires consideration of the validity of the instrument's content or its descriptive system, the valuations for the preference-weights and the empirical validity of the final preference-weighted scores. This paper aims to evaluate the validity of the CCI-6D descriptive system. The validity of the valuations and the empirical validity of the final preference-weighted scores are to be reported upon in a subsequent paper.
Study design
This study was nested in a large cross-sectional, observational study (INSPIRED) designed to evaluate the specialised dementia care currently being provided in Australian nursing homes. Participants were recruited from nine nursing homes across Australia, including both metropolitan and rural locations, over an eight-month period in 2015–2016. The study was approved by the Flinders University Social and Behavioural Research Ethics Committee (project numbers 6732 and 6753). Eligibility criteria included that the resident had been living in the home for at least 12 months, and was not currently receiving palliative care. Prior to completing the CCI-6D questionnaire, residents were administered the Psychogeriatric Assessment Scales – Cognitive Impairment Scale (PAS-Cog) by a trained research nurse, a cognitive functioning assessment tool with excellent reliability and validity, and widely used in the Australian long-term care sector for assessing cognition (Jorm et al. Reference Jorm, Mackinnon, Henderson, Scott, Christensen, Korten, Cullen and Mulligan1995). The PAS-Cog is scored on a scale between 0 and 21, where a higher score indicates greater cognitive impairment (Jorm et al. Reference Jorm, Mackinnon, Henderson, Scott, Christensen, Korten, Cullen and Mulligan1995). A score equal to or greater than 5 was shown to identify more than 80 per cent of dementia cases (Jorm et al. Reference Jorm, Mackinnon, Henderson, Scott, Christensen, Korten, Cullen and Mulligan1995). Scores can be further categorised to indicate no or minimal cognitive impairment (scores 0–3), mild impairment (4–9), moderate impairment (10–15) and severe impairment (16–21) (Department of Health 2017). Residents with no to mild cognitive impairment (indicated by a PAS-Cog score of between 0 and 9) were then asked to self-consent and self-complete the CCI-6D. Where residents had a more severe level of cognitive impairment (i.e. either indicated by a PAS-Cog score greater than 9 or judgement of a trained data collector that they did not understand the requirements of the study), a suitable proxy was approached to give informed consent. For those cases of proxy consent, questionnaires were either completed by the resident if able, or by a proxy completing on behalf of the resident in those cases where the resident had more severe cognitive impairment which prevented them from completing the questionnaire. For this study, a proxy assessor was defined as a person who had a close relationship with the individual, who visited the person regularly and assisted with making decisions on their behalf. This was usually a spouse, sibling or offspring of the individual. Following informed consent, participants took part in a face-to-face interview with trained data collectors. If a proxy participant was unable to attend a face-to-face interview, arrangements were made for them to complete the questionnaire via telephone or post. Data collected from the resident and/or proxy included the CCI-6D, quality of life (as measured by the Euroqol 5 Dimension 5 Level instrument (EQ-5D-5L) utility score; Devlin et al. Reference Devlin, Shah, Feng, Mulhern and Van Hout2016), dimensions of dementia-related quality of life as relevant (measured by the DEMQOL or DEMQOL-Proxy instruments; Mulhern et al. Reference Mulhern, Rowen, Brazier, Smith, Romeo, Tait, Watchurst, Chua, Loftus, Young, Lamping, Knapp, Howard and Banerjee2013) and social-care related quality of life (as measured by the Adult Social Care Outcomes Toolkit (ASCOT) – Care Home Version 3 Resident or Proxy instrument as required; Towers et al. Reference Towers, Smith, Palmer, Welch and Netten2016). Data collected from staff and records of the facility included physical function (Modified Barthel Index; Shah, Vanclay and Cooper Reference Shah, Vanclay and Cooper1989), frequency of access to ‘outside and gardens’ for the resident in the past week, and whether the resident had brought in their own furniture for furnishing their room, using standardised questions drawn from a previous study of nursing homes for people with dementia (Palm et al. Reference Palm, Kohler, Schwab, Bartholomeyczik and Holle2013). Hypothesis testing was used to assess the construct validity of the CCI-6D descriptive system and its expected relationships with variables (see Table 1). These expected relationships were drawn from evidence in the literature where possible, or from the opinions of experts in the research team in the area of person-reported outcomes, and aged care, following a previous methodology (Malley et al. Reference Malley, Towers, Netten, Brazier, Forder and Flynn2012).
Table 1. Variables included and expected associations
Notes: ASCOT: Adult Social Care Outcomes Toolkit. EQ-5D-5L: Euroqol 5 Dimension 5 Level instrument. CCI-6D: Consumer Choice Index – Six Dimension.
Statistical analysis
To assess for relationships between variables of interest, we used Pearson's chi-squared tests (for categorical variables) with use of related tests where appropriate (e.g. Yate's correction for continuity which compensates for the overestimate of the chi-square value in two-by-two tables). These tests have an assumption of minimum expected cell frequency (i.e. that 80 per cent or more of the cells should have a frequency of five or more). Where this assumption was violated, the maximum likelihood ratio chi-square test was used (McHugh Reference McHugh2013). The Kruskal–Wallis test was used to assess for relationships for continuous variables, with Mann–Whitney U-tests undertaken post hoc to test for differences between groups with Bonferoni adjustment. All analysis was undertaken using IBM SPSS Statistics Version 23. Associations significant at the 1 per cent level were taken to be highly suggestive of a relationship between the dimension and the variable. Associations at the 5 per cent level were taken to be weakly suggestive of a relationship between the dimension and the variable.
Results
A total of 661 residents were assessed for eligibility in nine nursing homes across five aged-care organisations in four states of Australia. Of these, 430 (65%) residents were deemed eligible and 254 (59%) consented to participate. Of those who declined participation, the most common reasons were being too busy to participate (26.69%) or a lack of interest in the study (19.32%). One participant did not undertake the CCI-6D questionnaire and was excluded from the analysis; 253 participants undertook a CCI-6D questionnaire and are included in the following analyses, of whom 68 (27%) were residents of long-term care facilities and 185 (73%) were proxy participants due to the severe cognitive impairment of the resident precluding their participation. Basic demographic characteristics of the sample are shown in Table 2.
Table 2. Socio-demographic characteristics of participants
Notes: N = 253. 1. Unless otherwise specified. EQ-5D-5L: Euroqol 5 Dimension 5 Level instrument. SD: standard deviation. PAS-Cog: Psychogeriatric Assessment Scales – Cognitive Impairment Scale.
For the CCI-6D questionnaire, 12 participants had one missing response and one participant each had two or three missing responses. These 14 participants were included in the study with a value imputed for the missing responses based on the mean of their responses to the remaining questions. The responses to the CCI-6D questionnaire are summarised in Table 3 for the total sample, and resident and proxy participants are presented separately. Over half of the participants reported they have the highest level of choice and flexibility in the dimensions of Care Time, Care Flexibility, Shared Spaces and Own Room. Participants indicated more variability and generally lower levels of perceived choice and flexibility in the dimensions of Outside and Gardens and Meaningful Activities. There were statistically significant differences in the distribution of responses between residents and proxy participants for the dimensions Own Room, Outside and Gardens and Meaningful Activities, with a higher proportion of proxy participants indicating lower levels of choice and flexibility in these areas. There were no statistically significant differences in the distribution of responses between proxy and resident participants for the other dimensions.
Table 3. Responses of participants to the Consumer Choice Index – Six Dimension (CCI-6D) questionnaire
Notes: 1. Pearson's chi-squared was used to generate p-values. 2. Or your family member (as appropriate for proxy participants).
CCI-6D and quality of life
We hypothesised there would be strong relationships between all of the CCI-6D dimensions and utility scores for social care-related quality of life (ASCOT) due to its focus on the long-term care setting, with relationships present for health-related quality of life (EQ-5D-5L) due to its generic nature and lack of specificity for the long-term care setting. This hypothesis was developed drawing upon the Donabedian model for assessing quality of health care, which proposed that care quality indicators fall into three main categories, i.e. those focusing on the organisational structures supporting care (structure), the processes through which care is provided (process) and the ultimate outcomes seen in recipients as a result of care (outcomes) (Donabedian Reference Donabedian1985). This model suggests that relationships between the three categories are linear in nature, i.e. that good structures of care will lead to good care processes, and that good processes for care will lead to good outcomes (Castle and Ferguson Reference Castle and Ferguson2010). This link has been further supported more recently, with improvements in outcomes (such as quality of life) identified following improvements to the structures or processes of care provided in nursing homes (Brownie and Nancarrow Reference Brownie and Nancarrow2013; Havig et al. Reference Havig, Skogstad, Kjekshus and Romoren2011; Kane et al. Reference Kane, Lum, Cutler, Degenholtz and Yu2007). On the basis of this previous evidence, we hypothesised that improvements in the processes of care, as assessed by the CCI-6D instrument, would be positively associated with higher levels of wellbeing and health and social care-related quality of life for care recipients (as the ultimate outcome of long-term care services). Overall, this hypothesis was supported (see Table 4), with strong relationships identified between social care-related quality of life (ASCOT) and all dimensions of the CCI-6D, and a strong relationship between health-related quality of life (EQ-5D-5L) and two dimensions (Outside and Gardens and Meaningful Activities). There was weaker evidence of a relationship between the EQ-5D-5L and the dimensions Care Time, Own Room and Care Flexibility. There was no evidence of a relationship between EQ-5D-5L and the Shared Spaces dimension. Post hoc tests identified the difference in EQ-5D-5L scores reported for those participants who selected the highest and middle response levels of the Care Time dimension reached statistical significance. Significant differences were also found in EQ-5D-5L scores for the participants selecting the highest and middle response levels of the Outside and Gardens and Meaningful Activities dimensions. Similarly, the difference in EQ-5D-5L scores reported for the highest and lowest response levels for the dimensions Care Time, Outside and Gardens and Meaningful Activities also reached statistical significance. Significant differences in the EQ-5D-5L scores were also found for the middle and lowest response levels of Meaningful Activities dimension, and the highest and middle response levels of the Care Flexibility dimension. Significant differences were identified between social care-related quality of life (ASCOT) scores reported for those participants who selected the highest and lowest response levels of dimensions Care Time, Shared Spaces, Own Room, Meaningful Activities and Care Flexibility. There was also a significant difference in ASCOT scores reported for the participants selecting the highest and middle response levels of the same dimensions. There were also significant differences in ASCOT scores for those selecting the highest and lowest and middle and lowest response levels of the Outside and Gardens dimension.
Table 4. Mean Euroqol 5 Dimension 5 Level instrument (EQ-5D-5L), Adult Social Care Outcomes Toolkit (ASCOT), Modified Barthel Index scores and age by Consumer Choice Index – Six Dimension (CCI-6D) dimension
Notes: 1. Or your family member (as appropriate for proxy participants). 2. Kruskal–Wallis test was used to generate p-values. SD: standard deviation.
CCI-6D and resident function and age
We identified the potential for a relationship between the CCI-6D and age and physical function (Modified Barthel Index). There was a strong relationship between responses to the dimensions Outside and Gardens and Meaningful Activities and physical function (Modified Barthel Index), and a weak relationship with Own Room dimension, but not with the remaining dimensions. Relationships were in the expected direction (i.e. a higher level of physical function for those indicating better access to Outside and Gardens and Meaningful Activities) and post hoc tests indicated significant differences in Barthel scores between the highest and middle and the highest and lowest response levels for both dimensions. Age did not show a significant relationship with responses to any of the dimensions, except for a weak relationship with access to Outside and Gardens.
The results of the tests of the hypothesised relationships between the individual variables and the dimensions of the CCI-6D are described below. The significance of the relationships between the six CCI-6D dimensions and the extent of control and flexibility in care, self-reported psychological wellbeing, social and occupational engagement, and the physical institutional environment are shown in Table 5, with the nature and direction of the association reported in the text.
Table 5. Significance of relationship between Consumer Choice Index – Six Dimension (CCI-6D) dimensions and associated quality of life, and model of care characteristics (p-values1)
Notes: 1. Pearson's chi-squared was used to generate p-values. ASCOT: Adult Social Care Outcomes Toolkit.
Care Time
As expected, we identified strong associations between the ASCOT Control dimension (p = 0.008) and the DEMQOL Social support dimension (p = 0.003) and the CCI-6D's Care Time dimension, with participants reporting the top level of Care Time generally also reporting better levels of control and social support. Strong associations were also shown between Care Time and ASCOT dimensions Social participation, Accommodation, Occupation and Dignity; DEMQOL dimensions Enjoying life; and DEMQOL-Proxy Hopes and aspirations for the future dimension, which were not hypothesised prior. There were no strong relationships between the Care Time dimension and physical care environment characteristics.
Shared Spaces
We identified a strong association between the CCI-6D Shared Spaces dimension and the ASCOT Accommodation (p = 0.001) and Social participation dimensions (p = 0.001). Relationships were in the expected direction, with higher levels of response to the home-like Shared Spaces associated with better responses on the ASCOT Accommodation and Social participation dimensions. We did not expect the strong associations that were found between the Shared Spaces dimension and the ASCOT dimensions of Control, and Dignity, the DEMQOL Enjoying life dimension, and the DEMQOL-Proxy Feeling useful dimension. There were no strong relationships between the dimension and any other variables.
Own Room
Residents bringing their own furniture in to furnish their room and the ASCOT Accommodation dimension were expected to be strongly related to the Own Room dimension. We identified strong relationships with the ASCOT Accommodation dimension (p ⩽ 0.001), but not with residents bringing their own furniture in to furnish their room (p = 0.377). Strong relationships were also found with the Own Room dimension and the ASCOT dimensions of Control and Occupation, and the DEMQOL-Proxy Feeling useful dimension, which were not previously hypothesised. Weaker associations were found with the Own Room dimension and the ASCOT Social participation dimension, DEMQOL Hopes and aspirations for the future dimension, and frequency of resident visiting the gardens.
Outside and Gardens
As expected, we found a strong relationship between the Outside and Gardens dimension and the frequency residents were reported to use the outside and garden area (p ⩽ 0.001). The relationship was in the expected direction (i.e. better responses to the Outside and Gardens dimension was associated with a higher frequency of access to outside and gardens by the resident). We also identified strong relationships between the CCI-6D dimensions and the Control, Social participation, and Occupation dimensions of the ASCOT, the DEMQOL Things that you want to do but can't dimension, and the DEMQOL-Proxy Hopes and aspirations for the future dimension.
Meaningful Activities
We identified strong relationships between the CCI-6D's Meaningful Activities dimension and the ASCOT's Control (p ⩽ 0.001), Occupation (p ⩽ 0.001), and Dignity (p = 0.004) dimensions. We also identified strong relationships between the dimension and the DEMQOL Enjoying life (p ⩽ 0.001), and DEMQOL-Proxy Hopes and aspirations for the future dimensions (p ⩽ 0.001). We did not find a relationship with the DEMQOL Things that you want to do but can't dimension (p = 0.391). We also found strong relationships that were not previously hypothesised between the CCI-6D dimension and the Social participation, and Accommodation dimensions (from the ASCOT), and the frequency of resident visiting the gardens variable.
Care Flexibility
We identified a strong relationship between the CCI-6D Care Flexibility dimension and the ASCOT Dignity dimension (p = 0.014), but no association between Care Flexibility and the Control dimension, or with the DEMQOL Social support dimension. However, we also identified a strong relationship between Care Flexibility and the ASCOT Accommodation dimension, and DEMQOL Enjoying life dimension, and a weak relationship with the DEMQOL Things that you want to do but can't dimension which was not hypothesised previously. The Care Flexibility dimension did not show a significant relationship with the other individual variables tested.
Discussion
This study has provided a preliminary assessment of the practicality and construct validity of the CCI-6D instrument for application in evaluating the quality of care of long-term care facilities from a consumer perspective. The vast majority of study participants were able to complete the CCI-6D instrument, and responses were dispersed across all three possible levels for each dimension comprising the instrument, demonstrating it is responsive to changes in the perception of care provision between individuals. In addition, there was evidence of a strong relationship with responses to the dimensions of the CCI-6D and social care-related quality of life, and a more moderate relationship with health-related quality of life, such that responses indicating greater flexibility and choice in care were associated with higher utility scores for the ASCOT and EQ-5D-5L instruments. This provides evidence of both the convergent validity of the CCI-6D instrument and of the clear associations first identified by Donabedian, between ‘process’ (reflected in the CCI-6D instrument) and ‘outcome’ (i.e. improved quality of life for individual residents) in defining the quality of care received in long-term care facilities from a consumer perspective.
Evidence has been presented for the construct validity of the CCI-6D dimensions (i.e. strong correlations with the expected variables) and discriminant validity (i.e. lack of relationship with variables not considered associated with that construct). Where relationships were not evident, reasonable explanations could be readily identified. We identified a strong correlation between the level of physical functioning of residents (as assessed by the Modified Barthel Index) and the dimensions of Outside and Gardens and Meaningful Activities, but not to the other dimensions. This finding may be explained by residents with greater physical functioning having greater access to outside and gardens and undertaking activities due to their greater independence, while those residents with functional impairments may find it more difficult to participate in activities offered as current practice in nursing homes. They may also be restricted in the activities they can undertake as a way of managing a risk of falls, or have lower levels of independence resulting in a reliance on others (e.g. staff) for access to the provided outside and garden areas. Given the high level of functional impairments by definition among residents, this level of dissatisfaction with these two dimensions highlights a need for action in these domains.
There are some limitations to this study which need to be considered. The study design facilitated the administration of the CCI-6D instrument to participants at a single time-point only. It was therefore not possible to assess the test re-test validity of the CCI-6D as a component of this study. Further research is needed to assess re-test validity and the responsiveness of the CCI-6D to change over time, a key requirement for assessing its potential for application as an outcome measure. We consider it very important to facilitate maximum participation of residents in evaluations of improvements to long-term care to ensure their voice is incorporated in design of services which directly impact upon them (Alzheimer Europe 2011; Castle and Ferguson Reference Castle and Ferguson2010). However, residents of nursing homes often are experiencing a variety of cognitive, sensory and functional impairments which impact on their ability to respond to questionnaires of this nature (Helvik et al. Reference Helvik, Engedal, Benth and Selbæk2015; Löfgren et al. Reference Löfgren, Bucht, Eriksson and Lundström1993). Therefore, an important consideration for future work will be developing administration methods to facilitate involvement of residents with a greater variety of cognitive or functional impairments, e.g. through use of visual representations of the dimensions and their respective levels. Methods such as talking mats or visual representations of dimensions and levels have shown promise in this regard and should be investigated further (Murphy et al. Reference Murphy, Tester, Hubbard, Downs and MacDonald2005; Turnpenny et al. Reference Turnpenny, Caiels, Crowther, Richardson, Whelton, Beadle-Brown, Apps and Rand2015).
In addition, as the majority of residents in long-term care facilities have functional and/or cognitive limitations, as expected a priori, a relatively large proportion of residents (75%) were unable to participate in the study themselves and therefore participation from a close relative was sought on their behalf. However, there are known systematic differences in the way that older people respond to questionnaires and how proxies respond even when they are asked to respond from the perspective of the older person (Alzheimer Europe 2011; Coucill et al. Reference Coucill, Bryan, Bentham, Buckley and Laight2001). For example, self-reported quality of life has been found to be significantly higher for people with dementia relative to proxy assessors (Addington-Hall and Kalra Reference Addington-Hall and Kalra2001). The current study represented the first application of the CCI-6D in practice. It is important to explore the reliability of this measure in participants with cognitive impairment, with a view to supporting their participation in evaluating quality of care in the sector. Future research should focus on assessing the extent of inter-rater agreement between residents and their proxies, as well as focusing on determining the range of residents with differing levels of cognitive impairment that can be supported to offer self (as opposed to proxy) assessment in completing the CCI-6D instrument. Currently, there is some evidence for thresholds for self-completion for instruments measuring quality of life such as the DEMQOL (Smith et al. Reference Smith, Lamping, Banerjee, Harwood, Foley, Smith, Cook, Murray, Prince, Levin, Mann and Knapp2007) and the EQ-5D (Hounsome, Orrell and Edwards Reference Hounsome, Orrell and Edwards2011), with these instruments showing acceptable reliability or completion rates among participants with mild or moderate cognitive impairment. However, the cognitive threshold for completion of an instrument is likely to depend upon its characteristics, including complexity of language and concepts presented, in addition to its method of administration (Sansoni et al. Reference Sansoni, Marosszeky, Jeon, Chenoweth, Hawthorne, King, Budge, Zapart, Sansoni, Senior, Kenny and Low2007). It can be postulated that the cognitive functioning threshold for self-completion may be lower for a simple and short questionnaire such as the CCI-6D, as compared to other more complex instruments.
Conclusions
The analyses presented in this paper present evidence of construct validity for the individual dimensions of the CCI-6D instrument, as assessed through the relationships with variables hypothesised to be associated with the dimensions of the instrument. Therefore, the CCI-6D shows promise in its ability to evaluate the quality of care processes from the perspective of consumers in long-term care facilities. The CCI-6D may be readily applied to evaluate the current levels of performance against key quality indicators identified by consumers for nursing home care (e.g. to provide a benchmark for the sector), or to evaluate the effect of systematic changes, interventions and quality improvement activities. As a measure of the quality of care processes in long-term care, the CCI-6D can be used in evaluations of innovations in long-term care to add to perspectives gained from the use of other more generically focused instruments, e.g. the ASCOT social care-related quality of life measure. In order to facilitate the potential for incorporation into an economic evaluation framework, the next stage of the research will involve the development of a preference-weighted scoring algorithm for application with the CCI-6D instrument.
Acknowledgements
The authors gratefully acknowledge funding provided by the National Health and Medical Research Council (NHMRC) Partnership Centre on Dealing with Cognitive and Related Functional Decline in Older People (grant number GNT9100000). The contents of the published materials are solely the responsibility of the Administering Institution, Flinders University and the individual authors identified, and do not reflect the views of the NHMRC or the Funding Partners. The authors sincerely thank the INSPIRED study participants and their family members for their participation and interest in the study. The assistance of facility staff, care-worker researchers and data collectors in each state is gratefully acknowledged. Members of the study team Mrs Anne Whitehouse and Dr Rebecca Bilton are thanked for their input into the study co-ordination and data collection. All authors contributed to the design of the study. MC, RM and CB contributed to the acquisition of data. RM analysed and interpreted the data and drafted the manuscript. All authors were involved in revising the manuscript and read and approved the final manuscript. The study was approved by the Flinders University Social and Behavioural Research Ethics Committee (project numbers 6732 and 6753). Written informed consent was obtained from all participants before interview. The authors declare that they have no competing interests.
Open access
