Medical devices encompass a wide range of nonpharmaceutical products used for treatment, alleviation, monitoring, or prevention of a disease or disability (1). This has become an increasingly important part of the healthcare system, and the market has grown considerably in recent years (Reference Sorenson and Drummond2;Reference Blixt, Södergård and Hiort3). The costs to the Swedish healthcare system have been estimated at over € 2 billion in 2013 (Reference Blixt, Södergård and Hiort3), which corresponds to approximately € 240 per capita. This is comparable to the costs of reimbursed medications in Sweden (Reference Blixt, Södergård and Hiort3).
There is a growing demand for health technology assessments of medical devices to assist the process of healthcare priority setting and decision making and in efforts to reduce regional inequalities in access to medical technology (4). However, the evidence standards concerning medical devices are generally less strict than for medications, and information about the devices’ efficacy and safety tends to be limited (Reference Sorenson and Drummond2). As a consequence, many medical devices are used within the healthcare system without a prior assessment of the cost-effectiveness (Reference Blixt, Södergård and Hiort3). In addition to the need for product assessments to support cost-effectiveness analyses, there is also a demand from the medical technology industry for early assessments of the potential value of a medical device from a regulatory, healthcare, and societal perspective (Reference Ijzerman and Steuten5–Reference Girling, Young and Brown7).
In addition to the limited access to efficacy and safety data, an additional challenge in health economic evaluations of medical devices is that the benefits of a medical device are often multifaceted. The value of considering the patient perspective has previously been acknowledged (Reference Blixt, Södergård and Hiort3;8;Reference Leidy, Beusterien and Sullivan9). For pharmaceuticals, the principally relevant property of a product from the perspective of patients and society is usually its pharmacological effect and the associated clinical outcome, but medical devices may provide important benefits to the patient beyond the clinical outcome, benefits that may have a large impact on the patients’ physical, mental, and social well-being. This includes product properties that impact an individual's everyday life and quality of life in a wider sense, such as relating to social aspects, integrity, and convenience. Today, there is no standardized method for measuring these benefits, which poses a challenge in evaluations of medical devices (Reference Blixt, Södergård and Hiort3).
Currently available instruments applied in health technology assessments, such as the EuroQoL 5-dimension questionnaire (EQ-5D) (10) and other generic instruments, provide methods to evaluate the clinical effect of a medication, but other aspects of importance for patients using medical devices are not directly captured. There are examples of instruments that measure such specific benefits for particular patient populations, such as patients with an ostomy (Reference Canova, Giorato and Roveron11), patients using urinary self-catheterization (Reference Pinder, Lloyd and Elwick12), or patients with diabetes (Reference Watkins and Connell13), but because these instruments are specific for a particular patient population they do not capture aspects that are common for patients using medical devices in general and are, therefore, less relevant for prioritizations across different groups of products. Patient benefits of medical devices may, therefore, be difficult to capture using the currently available health-related quality of life (HRQoL) instruments, and health economic assessments involving horizontal analysis (i.e., comparisons of products for different areas of use) are problematic.
To initiate the important discussion as to if and to which extent society should spend more resources on products which provide higher quality of life (i.e., more value) to the patient in the wider sense beyond clinical outcome, it is necessary to develop methods to measure these values. The objective of the current study was to explore and identify attributes of patient benefit that carry a general value to patients and other users of medical devices, with the aim to obtain a set of attributes that would be applicable in evaluations of all types of medical devices used by patients with any type of disease or disability. A further objective was to assess the relative preferences of these attributes in the general Swedish population.
METHODS
The exploration of attributes of patient benefit for users of medical devices followed a structured process including both qualitative and quantitative methods. An overview of the exploratory process is presented in Figure 1. The project was approved by the regional ethical review board in Gothenburg (Dnr 350-15). The authors assert that all procedures contributing to this work comply with the ethical standards of the relevant national and institutional committees on human experimentation and with the Helsinki Declaration of 1975, as revised in 2008.
Figure 1. Overview of the exploratory process.
Generation of General Attributes
A convenience sample of existing disease-specific HRQoL questionnaires (Reference Canova, Giorato and Roveron11–Reference Watkins and Connell13) was reviewed to explore benefits of potential importance for patients using various types of medical devices.
To identify general attributes of patient benefit that were relevant for various diseases and disabilities and for various medical devices, five in-depth face-to-face interviews were conducted with healthcare personnel (nurses and occupational therapists) working with a wide range of patient groups: patients having an ostomy, diabetes, urinary incontinence, or cognitive disorders, and elderly individuals with multiple diseases. The disease/disability areas were chosen to cover a heterogeneous group of products and for which medical devices were expected to have an impact on the patients’ physical, mental, and/or social well-being. Shortly after the interviews had been conducted, they were summarized and analyzed to find general attributes of patient benefit of potential relevance for various diseases and disabilities and for various types of medical devices.
The preliminary list of general attributes identified from the interviews were validated by means of a questionnaire distributed by e-mail to a reference group including the previously interviewed healthcare personnel (n = 5), and representatives for patient organizations (n = 5) and manufacturers of medical devices (n = 6). The respondents were asked to rate the relevance of each attribute on a four-graded scale, from “0” = not at all relevant, to “+++” = very relevant. Suggestions on additional attributes were also encouraged. The results were analyzed and the list of attributes was revised accordingly.
This revised list of preliminary attributes was evaluated in two consecutive pilot studies, conducted by means of Web-based questionnaires among respondents from the general Swedish population. Random samples of participants in a Web-based panel were invited to participate. In each pilot study, the invitations were distributed to a large sample and the data collection was closed once a predetermined number of responses had been collected. The panel included individuals who had previously participated in a nationally representative survey, and thereafter accepted an invitation to join the panel. In these pilot studies, the respondents were asked to rank how they perceived the importance of each attribute for individuals using medical devices (i.e., rather than considering their own needs or health status).
To ascertain the understanding and relevance of the attributes, two rounds of cognitive interviews were performed according to standard methodology (Reference Beatty and Willis14). For each round, five respondents were recruited from the general population by means of convenience sampling. During face-to-face interviews, all attributes were presented in writing and the respondents’ interpretation of the content and the clarity of wording were noted by the interviewer. In addition to testing the attributes, the understanding of the background questions and the instructions that would be used in the preference-elicitation questionnaire (described below) were also tested during these interviews.
The attributes were finally allocated into domains by the research group.
Elicitation of Relative Preference of Each Attribute in the General Population
With the purpose of eliciting the relative preferences of the attributes in the general population, a Web-based questionnaire was developed. The questionnaire was designed as a ranking exercise so that respondents were required to prioritize and choose between attributes, and hence yield preferences. Instead of simultaneously ranking all attributes, the attributes were randomly distributed into five equal groups and ranked separately within each group, because ranking exercises including a large number of items were expected to be of limited validity. The order in which the attributes were presented was randomized, to avoid potential response bias due to item order. The respondents were asked to rank the attributes based on how they perceived their importance for individuals using medical devices. Medical devices were described as nonpharmaceutical products used for, for example, treatment, alleviation, monitoring, or prevention of a disease or disability. The questionnaire also included background questions on the respondents’ sociodemographics and health status.
A usability test of this Web-based questionnaire was carried out including five face-to-face interviews with respondents from the general population selected by means of convenience sampling.
The usability-tested questionnaire was distributed to 15,000 participants (18 years or older) of a Web-based panel (the same panel as used in the pilot studies). A sampling strategy was applied to obtain a representative sample of the general Swedish population with respect to age, sex, and residential region, based on population statistics from Statistics Sweden (15). This yielded twenty different strata based on age (five groups), sex, and region (resident within a metropolitan region [one of the three largest cities in Sweden] or not). The required number of responses within each strata corresponded to the distribution in the general Swedish population. The strata were kept open throughout the entire survey period, meaning that strata were allowed to become overfilled (i.e., >100 percent of the required responses) in the data collection phase. After data collection, excess respondents in each of the overfilled strata were excluded at random so that the final age-, sex, and regional distribution of respondents corresponded to that in the general Swedish population. This strategy was applied to avoid the risk of early-responder bias.
Based on the responses from the questionnaire, the perceived relative importance of the attributes was assessed in the overall sample, and also compared across subgroups based on the respondents’ background characteristics.
Statistical Analyses
Continuous variables are presented with mean and standard deviation (SD). Categorical variables are presented with frequency and percentage. Percentages are based on the number of nonmissing observations.
Based on the data collected from the preference-elicitation questionnaire, relative preferences of each attribute were calculated using a statistical model (a normal score ranking model). This model was adapted to convert the data from the preference-elicitation questionnaire (consisting of impartial ranks of the attributes; because each respondent ranked four attributes at a time instead of ranking all twenty attributes simultaneously), into individual preference scores for all attributes. The model yielded the relative distances between the attributes’ scores, reflecting the probabilities that one attribute would be ranked above another in a given ranking exercise between them. Indifference between two attributes would thus result in identical preference scores. The relative distances in preference score between the attributes were used to generate a relative preference of each attribute (i.e., the relative importance of each attribute).
Rankings between subgroups of respondents were also compared using the statistical model. Potential differences in rankings of attributes between subgroups were assessed using two-tailed Z-tests. The significance level was set to p < 0.001 to adjust for the large number of statistical tests performed.
Statistical analyses were performed using SAS version 9.4 (Cary, NC).
RESULTS
Development and Validation of the General Attributes
To identify general attributes of patient benefit, healthcare personnel were interviewed in depth. Five interviews were performed, each lasting 1 hour. The interviewees had on average 15.8 (SD 9.0) years of experience from working with medical devices within their disease/disability area. The average number of patient contacts per week was 14.8 (SD 12.7). The findings from these interviews resulted in a preliminary list of thirty-one general attributes.
To assess the validity of the thirty-one preliminary attibutes, a questionnaire was distributed to the reference group by means of e-mail (n = 16). The overall response rate was 81.3 percent. Their input on the relevance of the attributes and suggestions on additional attributes resulted in the deletion of eight attributes, the addition of one attribute, and the clarification of several of the remaining attributes. After this validation step, there were twenty-four general attributes.
To assess the validity of the attributes in the general population, two consecutive Web-based pilot studies were conducted. The first pilot study included 119 respondents, and the second pilot included 260 respondents. Because the data collection was closed once the sufficient number of responses had been collected, response rates were not assessed. Based on the first pilot study, minor changes in the wordings of the attributes were made to improve the understanding. Overall, the results from the second pilot study were similar to those from the first pilot study.
Two rounds of cognitive interviews were performed to ascertain the understanding and relevance of the attributes. The first five cognitive interviews lasted 50 minutes each, on average. The findings from these interviews resulted in clarified wording and the exclusion of four partially overlapping attributes. Thus, twenty general attributes remained. The five interviews in the second round lasted approximately 30 minutes each, and the input from these interviews resulted in minor revisions in wording for some of the attributes.
The development and validation process thus resulted in twenty general attributes of patient benefit of medical devices. The attributes were allocated into the following domains: sense of security, social participation, integrity, and convenience; each domain including five attributes. The attributes and the domains are presented in Figure 2.
Figure 2. The resulting twenty general attributes.
Preference-Based Weights of Each Attribute
Background Information
The general population's preferences of the attributes were measured with a Web-based usability-tested questionnaire distributed to a representative sample of 15,000 individuals. Responses were provided by 5,545 individuals, rendering an overall response rate of 37.0 percent. Excess respondents in each of the overfilled strata were excluded at random so that the final distribution of respondents corresponded to that in the general Swedish population with respect to age, sex, and residential region. The final population consisted of 3,802 respondents.
The characteristics of the final population are presented in Table 1. In brief, approximately one-third (31.9 percent) were aged 60 years and older, and 50.4 percent were women. The vast majority of the respondents (75.3 percent) rated their health status as good or very good. One-third (31.6 percent) reported personal experience from a long-lasting disease, and one-fifth (19.9 percent) had personal experience from a long-lasting disability. One-third (33.3 percent) reported to have personal experience from using a medical device.
Table 1. Characteristics of the final study population (n = 3,802)
Note. Percentages are based on number of non-missing participants.
a EUR 1 = SEK 9.36 (2015 average exchange rate)
The respondents took an average 8.5 (SD 5.2) minutes to complete the preference-elicitation questionnaire. Computers were most commonly (63.4 percent) used for responding to the questionnaire while 36.6 percent responded by means of a mobile phone or a tablet.
Perceived Relative Importance of the Attributes
The results from the preference-elicitation questionnaire showed that the attributes with highest relative preferences in the Swedish general population were attributes related to perceived reliability of the medical device, reduced need for assistance from others, sense of control of the illness/disability, and reduced sense of being ill/having a disability.
Analyses were performed to compare the perceived relative importance of attributes between subgroups of the study population. As presented in Table 2, no statistically significant differences were observed between subgroups based on type of housing, household composition, household income, education level, self-perceived active life, self-rated health status, long-lasting disability, or self-reported use of a medical device. On the other hand, differences were observed between subgroups based on age, sex, and experience from having a long-lasting disease.
Table 2. Subgroup analyses of perceived relative importance of the attributes
a Attributes with a statistically significant (p < 0.001) difference in perceived relative importance between the compared subgroups.
b EUR 1 = SEK 9.36 (2015 average exchange rate).
Female respondents perceived attributes concerning personal hygiene and adaptability to personal needs to be more important than male respondents did.
Younger respondents perceived attributes relating to integrity, social participation, and reducing discomfort at use to be more important, while older respondents perceived facilitation of personal hygiene, adaptability to personal needs, and attributes relating to sense of security (aid to remember tasks and feedback that the device is used correctly) to be more important.
Finally, compared with respondents with experience from having a long-lasting disease, respondents without such experience perceived devices that reduce feelings of compromised personal integrity (someone getting too close) to be more valuable.
Design of a Patient-Reported Questionnaire Based on the Attributes
The development process resulted in the design of a patient-reported questionnaire involving the twenty attributes, designated the MedTech20 Questionnaire, and intended for use in evaluations of medical devices. The questionnaire was designed as an assessment of how well the evaluated medical device fulfills each of the twenty general attributes on a seven-graded scale spanning from “not at all” to “completely.” In order for the questionnaire to be applicable in different stages of a device's life cycle, different versions were formulated, allowing the respondents’ assessment to be based either on personal experience from using the device(s) under study, or on a standardized description of the device's function and design.
For information on how to access to the MedTech20 Questionnaire, in English translation or in Swedish original, please consult www.medtech20.com. Intellectual property: The MedTech20 including but not limited to all and any translations and other derivatives (e.g., electronic versions) is protected by international copyright with all rights reserved to NHE Licence AB [US Copyright Office Preregistration Number: PRE000008511]. Do not use without permission. For information on, permission to use or translations of the MedTech20 Questionnaire, please consult www.medtech20.com.
DISCUSSION
This report presents the exploration of attributes describing the potential benefits of medical devices, from the patient's perspective, formulated on a generalized level, and applicable in evaluations of devices used in any type of disease or disability. The development process rested on a review of disease-specific questionnaires, and input from healthcare personnel, patient organizations, manufacturers of medical devices, as well as the general population. It resulted in the identification of twenty general attributes of patient benefit relevant for users of medical devices. The relative importance of each attribute was then explored from the preferences of the general population in Sweden as measured in a large-scale survey.
Potential patient benefits of medical devices were identified based on a review of existing disease-specific HRQoL instruments and in-depth interviews with experienced healthcare personnel. Because the intention was to develop a generic questionnaire applicable for patients with any type of disease or disability, the patient population for which the questionnaire was intended is very heterogeneous. The rationale for initially interviewing healthcare personnel as opposed to patients was that healthcare personnel were expected to possess a wider knowledge base of perceptions among various patients, as compared to interviewing individual patients. The interviews were analyzed to find general attributes of patient benefit with potential relevance for various diseases and disabilities, and for various types of medical devices.
A reference group including healthcare personnel and representatives from patient organizations and manufacturers was consulted to assure that the attributes maintained their relevance also when generalized, and to potentially identify any additional attributes of importance. Subsequent pilot studies showed that the perceived relative importance of the attributes in the general population was consistent with the expected pattern. The attributes were also subject to two rounds of cognitive debriefing interviews. Based on the findings from these validation procedures, the face and content validity of the attributes was considered satisfactory.
The structured exploration process resulted in the identification of twenty attributes that were found valid, relevant and important for patients using medical devices. They included different aspects of how well a medical device affects the patient's well-being (e.g., reduced sense of being ill/having a disability, facilitation of personal hygiene, reduced barriers to a good sleep), of how the device is perceived by the patient (e.g., perceived safety leading to less worry), and of product usability (e.g., ease of use, adaptability to personal needs).
It was an aim of the exploration process to identify potential and desired product attributes from the patients’ perspective, and a further aim that these attributes expressed benefits on a general level rather than on a product or disease specific level. The resulting list of attributes forms a map of the spectrum of product attributes that are valuable for the patients, but it does not necessarily mean that every attribute is relevant for all possible medical devices. This might not be possible to achieve, much alike evaluations of pharmaceuticals using the EQ-5D instrument, for example, where all pharmaceuticals do not necessarily impact every attribute measured by the instrument. Similarly, certain diseases are associated with much higher quality-adjusted life-year (QALY) loss than others, and thus provide a much larger potential value in terms of QALY gain for treatments of these diseases compared with treatments of less severe diseases.
The preferences of the attributes were elicited in a large sample which was representative of the general Swedish population with respect to age, sex, and residential region. The results showed that the attributes with the highest reported preferences concerned reliability, reduced need for assistance, and sense of control of the illness/disability.
While the education level in the sample was somewhat higher than in the general Swedish population (proportion with university or university college education: 58 percent versus 34 percent) (16), the self-reported health status in this sample was similar to that reported in the Swedish Living Conditions Survey, which is nationally representative (17). The proportion reporting a good or very good self-reported health status in the sample was similar to that in the Living Conditions Survey (75 percent and 78 percent, respectively, and the proportion with experience from a long-lasting disease was 32 percent and 37 percent, respectively).
Experience from using a medical device was common in the general population; this was reported by one third of the respondents. There were no statistically significant differences in preferences for the attributes between respondents with and without personal experience from using medical devices. This supports that the views in the general population are relevant also for the population in need of medical devices, and that it can be justified to base healthcare decisions and prioritizations on the general population's preferences on this topic. The intuitive expectation that attributes concerning social participation and integrity would be perceived as more important by younger compared with older individuals was indeed confirmed by the results from subgroup analyses. In further alignment with expected outcomes, older respondents gave a higher rank to attributes concerning aid to remember tasks in relation to the disease or disability, and feedback mechanisms for informing about correct use of the device.
The twenty attributes were categorized into the following domains: sense of security, social participation, integrity and convenience. Because the data were collected based on prioritizations between attributes (ranked data) and thus internally dependent, factor analysis was not considered appropriate to perform. Instead, the domains were formed merely for presentation purposes and not with the aim of statistical comparisons. The allocation of attributes into domains was made by the research group based on semantic grounds.
A formalized questionnaire, designated the MedTech20 Questionnaire, was designed based on the identified set of attributes. It is worth noting that the purpose of this questionnaire was not to provide a complete method for health technology assessment of medical devices. Many aspects of the utility of medical devices that should be taken into account in a health technology assessment can be satisfactorily evaluated with currently available methods, such as improved clinical outcome as measured in terms of QALYs, or resource savings in the healthcare setting. However, potentially important aspects of the device which affect the patient's everyday life are currently not included and hence not valued in health economic evaluations, which hamper the incentives to develop devices that provide such benefits to the patients. We believe that the reason for this current societal prioritization is not a lack of willingness to pay for such benefits, but rather a lack of methods to quantify such benefits. This view has been endorsed by both national and regional payor organizations. We believe that an assessment with the questionnaire designed based on our current results would provide an additional and important piece of information missing today: the benefits of a medical device from the user's perspective.
A common limitation of patient-reported instruments and questionnaires are their validity in different health systems. Regarding the MedTech20 questionnaire, it provides answers to the question of how a medical device is perceived by the user. We believe that this question is less dependent on the structure of the healthcare system, and the results should be possible to transfer across national borders. Naturally, the preferential ranking of attributes may differ between countries, so exploring nation specific preferences for various countries would be optimal, and is a scope of future research. Nonetheless, results from evaluations based on preferences explored in other countries are commonly used in health economic evaluations (e.g., the British tariff for EQ-5D health states) (Reference Dolan18).
The main strength of the set of attributes identified in this study is that they provide the basis for a generic method to evaluate potential patient benefits associated with medical devices, relevant for a wide range of diseases and disabilities. To our knowledge, no such general method is yet available in the scientific domain. The questionnaire that was designed based on the identified attributes is applicable for evaluations of all types of medical devices used by patients with any type of disease or disability, and it can be applied in various phases during a device's life cycle. The questionnaire may also be applied during early product development to assess the potential value of a medical device from a regulatory, healthcare, and societal perspective. A limitation of the study was that convenience sampling was used to select a sample for review of existing disease-specific HRQoL questionnaires as well as for selecting respondents for cognitive interviews during the validation of the identified attributes. Meanwhile, the power of the population size in the preference-eliciting survey was a strength of the study.
Policy Implications
The MedTech20 Questionnaire provides a new method for assessment of patient benefits associated with medical devices. As a disease generic questionnaire, it offers a possibility to compare the patient-perceived value between products as well as across product categories and, thus, can be used to supplement health economic assessments used for vertical analysis; comparison of products for similar use, as well as horizontal analysis; comparison of products for different areas of use. Hopefully, this will contribute in addressing the high unmet need within this area and assist in future healthcare decision-making.
An additional potential use of the MedTech20 Questionnaire would be in public tendering processes, where it could serve as the basis for quality criteria where, for example, a threshold could be defined which products must pass to be subject for further evaluation in the tender.
CONCLUSIONS
This study identified and validated a set of twenty attributes of patient benefit potentially provided by medical devices. The most highly preferred attributes in Sweden's general population concerned reliability, reduced need for assistance, and sense of control of the illness/disability. A questionnaire for patient-reported assessment of the benefits provided by a medical device was designed, based on the identified attributes. This questionnaire provides a generic measurement method for the evaluation of medical devices used in a wide range of diseases/disabilities.
Open access
