Introduction
Understanding the perspectives of patients and their families and carers is becoming widely recognized as a critical component in any Health Technology Assessment (HTA) for reimbursement purposes (1–Reference Weeks, Polisena, Scott, Holtorf, Staniszewska and Facey3). Patients, families, and carers can provide unique information about what it is like to live with a condition and give a real-life view of the potential impact of a new medicine (Reference Perfetto, Oehrlein, Boutin, Reid and Gascho4). Patient groups can help provide this information through their engagement with the HTA process, which is illustrative of inclusivity, a core principle of effective deliberative processes in HTA (Reference Bond, Stiffell and Ollendorf5). Inclusivity involves creating an environment where stakeholders can meaningfully contribute to the HTA decision-making process, for example by sharing information and/or providing this information in plain language. Often, the content of an HTA submission is inaccessible or quite technical. As a result, patients or patient groups may find it difficult to comment on the proposed use of a treatment or discuss trade-offs of the benefits and harms of a treatment. HTA bodies, however, often ask patients or patient groups these questions to inform deliberations in addition to questions about patients’ experience of living with a condition (Reference Single, Facey, Livingstone and Silva6–Reference Sampietro-Colom and Thomas8). It is, therefore, important that relevant patient audiences have an informed and appropriate understanding of the treatment or technology under review in order to optimize their input into the HTA process.
Many HTA bodies emphasize the involvement of patient groups and have developed working practices to solicit input to inform these deliberative processes (Reference Facey, Hansen and Single9). These include the use of consultation surveys and consultation documents. These also include the initiative of the Scottish Medicines Consortium (SMC) to develop a template for pharmaceutical companies to complete that could be provided to patient groups as a plain language summary of the medicine under appraisal (Reference Single, Macpherson, Fearns, Dickson, Facey, Facey, Ploug Hansen and Single10). In 2017, the SMC introduced an update to this template in the form of the Summary Information for Patient Groups (SIP) (11). This later became a mandatory part of HTA submissions to the SMC. However, most other HTA bodies have yet to follow the example of the SMC, and patient input to the HTA process is often in the absence of or with limited information about the intervention (drug, diagnostic, or device) under evaluation. Pharmaceutical companies making an HTA submission to the SMC are asked to provide answers to seven questions, covering the following: the use of the medicine, management of the condition in Scotland, how the medicine works, its effectiveness and safety, and the impact of the medicine on quality of life for patients and carers. The SIP is made available by the SMC as a resource to support the SMC Patient Group Submission process (12).
The Health Technology Assessment international (HTAi) Interest Group for Patient and Citizen Involvement in HTA (PCIG) project was initiated in response to recognition of the value of providing patient groups with clear information, and to help provide tools to support and encourage robust and meaningful involvement of the patient perspective in HTA deliberations. Two specific objectives of the project were (1) to collect qualitative data to assess the experience of using the SMC SIP and (2) to develop an international SIP template based on these findings.
Here, we present the results of the qualitative evaluation of the SMC SIP from pharmaceutical company representatives, HTA bodies, and patient groups. Data collection methods consisted of the following: a survey of pharmaceutical representatives on their experience of using the SMC SIP; a survey of patient groups on their experience of receiving completed SIP documents as part of HTA applications in Scotland; semistructured interviews with representatives of HTA bodies; and discussions with patient groups in the United States, Australia, and Europe to gather feedback on the SIP approach more generally. These activities were conducted by the HTAi PCIG project team. Formal research ethics approval was not sought for these activities as this was considered a quality improvement project. These qualitative data were used to inform the development of an international SIP template and associated guidance documents. Although various “lay summary” formats exist, the SMC SIP was chosen as the basis for the International SIP template as it is already in use in Scottish HTAs and is known by pharmaceutical companies that would need to accept this as an additional part of their submission process. Furthermore, patient groups have found the format to be helpful (Reference Single, Facey, Livingstone and Silva6). Further information regarding the SMC SIP can be found under Section 8 of the New Product Assessment Form at: https://www.scottishmedicines.org.uk/making-a-submission/ and in the Guidance to Manufacturers document at: https://www.scottishmedicines.org.uk/media/2771/guidance-on-summary-information-for-patient-groups.pdf.
Methods
Surveys of Pharmaceutical Company Representatives and Patient Groups
Two online surveys were designed by members of the SMC public involvement team in consultation with the HTAi PCIG project team to obtain feedback from pharmaceutical companies and patient groups on their experience of either completing or receiving an SIP relating to medicines submitted to the SMC for appraisal between October 2018 and November 2019. The two e-surveys were developed using SmartSurvey build version 5.5.0.1809 and consisted of a mix of open and closed questions (e-surveys available in the Supplementary Material).
The e-survey of pharmaceutical company representatives consisted of thirteen questions, and respondents were advised that it would take approximately 10 minutes to complete. Closed questions on simple agreement scales asked respondents to assess: whether pharmaceutical companies value SIP forms; whether the SMC SIP form resulted in better-informed patient input; whether it was easy to complete; the sufficiency of the guidance provided for content and writing style. Respondents were also asked whether and how they assessed the readability of the SIP form. Open questions asked respondents to expand on these answers and suggest any improvements to the SIP.
For the e-survey of patient group representatives on the experience of receiving the SMC SIP, closed questions asked respondents to assess: whether they read the form and found it helpful; how helpful they found each section of the SMC SIP; whether the information was at the right level of detail; ease of completion and clarity of language; and whether the information in the form was credible. Open questions asked respondents to expand on these answers. The patient e-survey consisted of twelve questions, with respondents being advised that it should take approximately 10 minutes to complete.
A link to each e-survey was disseminated by the SMC by e-mail in November 2019 to pharmaceutical company representatives (n = 32) and patient group representatives (n = 55). A 3-week period was allowed for completion of the surveys. Responses were anonymous. Descriptive statistics were used to analyze responses to the closed questions. Responses to open questions were used to understand the reasoning behind answers and to explore themes relating to the user experience.
Semistructured Interviews with HTA Body Representatives
Experience of implementing the SIP from an HTA agency's perspective was collected through semistructured interviews with staff from the SMC who had been involved with the SIP process. The semistructured interview guide was developed by the HTAi PCIG project team and covered the rationale for introducing the SIP, perceived or expected benefits of the SIP, and barriers or challenges associated with the implementation of the SIP.
Further interviews were conducted with senior representatives and/or staff at the National Institute for Health and Care Excellence (NICE) in England, the Pharmaceutical Benefits Advisory Committee (PBAC) Secretariat in Australia, and Aragon Health Sciences Institute (IACS) in Spain. These interviews focused on understanding the feasibility of implementing an SIP in the local context and potential barriers and challenges to implementation.
Discussions with Patient Groups
Patient/consumer group discussions were held in Europe, Australia, and the USA to explore the value of the SIP approach for patient groups and to examine what information would be considered useful for these groups to assist with HTA participation.
Discussions with patient groups in Australia and the USA were connected to formal patient group meetings that were already planned. Two example-completed SIPs were shared in advance of these meetings to facilitate discussions. Patient representatives provided feedback regarding the content of the SIP template (e.g., what was helpful, what could be added, etc.), language (e.g., tone, perceived bias, readability, etc.), and other comments regarding the use of the SIP in their respective countries. For the U.S. meeting, two SMC staff members joined virtually to introduce the SIP template and answer questions about how the SMC uses the SIP as part of an HTA assessment. European groups were contacted and provided informal feedback via e-mail or interviews. Prompting questions for discussion included whether SIP forms provide useful information, whether any key information was missing from example SIPs, and how SIPs might be improved. Sampling for all groups was conducted on a convenience basis, based on existing networks to provide input from a variety of patient groups from three different regions of the world.
Results
Evaluation of the SMC SIP Template and SIP Approach
Survey of Pharmaceutical Company Representatives
Out of thirty-two pharmaceutical company representatives who received the survey link from the SMC, twenty-two responded to the questionnaire (response rate: 69%). Most industry respondents (18/22, 82%) agreed that completing the SIP form was a worthwhile investment of their time, with four respondents (18%) indicating that they partially agreed. When asked if the SIP form resulted in more informed patient input to the SMC's appraisal process, 62 percent (14/22) agreed or partially agreed, 27 percent (6/22) stated that they did not know, and some explaining in free text that they had not received feedback from patient groups and that this would be beneficial in the future. The respondents who did not agree (2/22, 9%) gave no explanation for their disagreement.
Almost all respondents (21/22, 95%) agreed that the need for the SIP was clearly explained. Overall, 73 percent (16/22) of respondents reported that the SIP form was “very easy” (3/22, 14%) or “easy” (13/22, 59%) to complete. The remainder (6/22, 27%) reported that the form was “Neither easy nor difficult.” Although none of the respondents reported that it was difficult to complete the SIP form, free-text responses highlighted that writing in plain English can be challenging (Table 1).
Table 1. Direct quotes from e-survey of pharmaceutical company representatives’ perspectives on the SMC SIP form
Respondents agreed (19/22, 86%) or partially agreed (3/22, 14%) that there was sufficient guidance on how to complete the SIP in terms of content, and agreed (14/22, 64%) or partially agreed (8/22, 36%) that there was sufficient guidance in terms of writing style/level. Suggestions for additional guidance included provision of best practice example SIP forms, advice on producing lay documents, and a glossary of terms. Most respondents (16/22, 73%) reported that there was no “plain English check” of SIP documents when they were drafted (e.g., with a readability assessment such as the Flesch reading ease test (Reference Spadaro, Robinson and Smith13)).
When asked how the SIP form could be improved, the majority of respondents reported that the content of the SIP form was appropriate (19/22, 86%). Two respondents (14%) reported that additional information was required and suggested including more information on cost-effectiveness and statistics to help the patient representatives interpret the data.
Survey of Patient Group Representatives
Out of fifty-five patient groups contacted by the SMC, seventeen survey responses were received (response rate: 31%). Of these, most (15/17, 88%) reported that they had read the information in the SIP form. The respondents who did not read the SIP form (2/17, 12%) explained that they prefer to conduct their own research and refer to independent sources as they believe that the information in the SIP form would bias their submission to the HTA. Of the fifteen who reported reading the SIP, ten (67%) reported finding the document “helpful” and five (33%) reported it to be “very helpful.” Open question responses suggested that the SIP form was used in different ways, including: to improve understanding of the technical aspects of a new medicine; to complement existing knowledge; to gain insight into the company viewpoint; and to inform communication with patients/families when eliciting their perspectives.
Each section of the SIP form was considered “very helpful,” “helpful,” or “slightly helpful” by all respondents; no sections were identified as “not at all helpful.” When asked which section was the most helpful, individuals’ free-text responses were highly varied (Table 2).
Table 2. Direct quotes from e-survey of patient groups’ perspectives on the SMC SIP form
In general, patient groups felt that the format of the SIP form was easy to use and that the information provided was at an appropriate level of detail, although opinions were mixed regarding understandability. Although many patient group respondents agreed that the terminology used in the SIP form was well explained, others felt there was still too much technical language and abbreviations. One respondent noted that too much scientific language could potentially put people off reading the SIP.
In total, seven respondents made suggestions for additional content for the SIP form, including: information on where the new medicine belongs in the existing treatment pathway; details of when the medicine would be made available; and information on the side effects of the new medicine compared with existing treatment(s).
Semistructured Interviews with HTA Body Representatives
Representatives of the SMC generally reported positive feedback from implementing the SIP and felt that it had improved patient group involvement in the HTA process, with the number of submissions with input from patient groups increasing along with an overall improvement in the quality of the patient group submissions.
Prior to the development of the SIP, the SMC asked patient groups to complete submission forms, but did not provide information about the medicine being assessed. Representatives of the SMC reported that some patient groups found this a challenge and that the development of the SMC SIP had helped overcome this issue. Furthermore, the SIP had led to increased satisfaction among patient groups and a greater feeling that they were being supported by the SMC.
It was noted that potential concerns with the approach had been that patient groups could infer that an SIP document may contain biased information because it had been drafted by pharmaceutical companies, and that patient groups would replicate information provided rather than using the information as a basis to inform their own submission as to what matters most to patients. It was reported that this had occurred only in a small number of cases and that where it had occurred, the SMC public involvement team, as part of their review process of the SIP and patient submissions, had asked for those documents to be revised.
Interviews with representatives from HTA bodies on the feasibility of the SIP identified several aspects to be considered when implementing it. These considerations included the importance of establishing agreement from the relevant industry body, understanding where the SIP would fit into the sequencing of the HTA process, who would be responsible for the content, and what resources would be needed to implement the SIP locally (Table 3).
Table 3. Aspects for HTA agencies to consider when implementing the SIP
Feedback from Patient Group Sessions
Patient group feedback was provided by representatives from 6 European patient groups, representatives from 18 U.S. patient groups, and representatives from 12 Australian patient groups. Table 4 summarizes feedback provided by the patient groups. Overall, patient groups liked that the SIP documents were concise, used a question and answer approach, and were formatted with bullet points and/or tables.
Table 4. Summary of feedback from patient groups on the summary of information for patients template on potential information and layout
Some patient representatives, however, commented that the documents were still quite dense and noted that some patient groups would not have the level of expertise and/or health literacy required to interpret these documents. To assist with understanding, diagrams and infographics could be used, and readability assessments could be undertaken to check complexity in addition to a patient reviewer prior to it being distributed.
Some patient groups also suggested that an executive summary was needed at the beginning of the document to help highlight key points prior to reading and that a glossary of terms would be useful. Patient groups also wanted to know how the document would be distributed, whether it would be available to individual patients (as distinct from patient groups), and what role the HTA body would have in reviewing the content.
Development of an International SIP Template and Preparation for Local Implementation
The findings from the project-related activities were used to draft an international SIP template consisting of four key sections: (1) disease background information; (2) information on the new treatment and its potential therapeutic benefits; (3) information on the economic value of the new treatment; (4) further resources that may be of interest to the reader. Compared with the SMC SIP template, the proposed international version includes key changes that can be summarized as follows: inclusion of an economic value section, a patient-based evidence section, a glossary, and generalization beyond Scotland. The qualitative findings from the survey and patient group sessions were also used to refine the language used in the template and to develop guidance documents to support industry, HTA bodies, and patient groups in the completion, assessment, and use of the SIP. These documents, along with the International SIP template (Version 1), can be found on the HTAI Web site (https://htai.org/interest-groups/pcig/projects/current-projects/).
Discussion
The importance of the patient perspective in HTA is increasingly appreciated, and the growing participation of patient groups in HTA in some countries has shown positive effects. Nevertheless, there remains a need for improvements and consistency in approaches used, taking into account both common (e.g., time and resource constraints, “real-world” practicalities, etc.) and country-specific challenges (Reference Hailey, Facey, Ploug Hansen and Single14;Reference Wong-Rieger, Facey, Ploug Hansen and Single15). The international SIP template is intended to become a consistent information resource available to support patient and patient group input into HTA, which can be shared by HTA bodies and completed by the industry as part of their submissions.
HTA submission documents are frequently written using technical terms that may not be understandable to lay audiences. Although the current SMC SIP received positive feedback, a few of the patient groups involved in this project reviewed past SMC SIPs and considered the language overly technical. Provision of a suitable SIP template may help standardize language at a more appropriate level for patient groups. This also raises the need for industry to develop the capabilities to write for a lay audience and/or for HTA bodies to provide training and support in this area. The qualitative results collected here suggest that further guidance on the use of plain language may help (such as more detailed guidance documents, provision of a glossary, and the use of readability assessments and a lay/patient reader).
Regardless of these initiatives, some patient groups may choose not to use an SIP. Our findings suggest that this may be because some groups are comfortable undertaking their own research of the medical literature, and/or are familiar with the medicine under evaluation, or have concerns around the credibility and potential bias associated with information being provided by pharmaceutical companies. Given that patient advocacy groups are highly heterogeneous, this is expected. There are significant differences in background, access to resources, expertise, and experience with HTA processes among patient groups, as well as differing attitudes toward pharmaceutical companies. HTA bodies are best placed to undertake activities that support impartiality, transparency, and inclusivity. In implementing the international SIP template, this may involve providing regular guidance, training, and feedback to all stakeholders, as well as implementing a formal review process of the SIP prior to distribution to patient groups.
One key challenge for the creation and standardization of an international SIP template is that the provision of information to patient groups by pharmaceutical companies will be subject to different regulations in different countries (this is emphasized in draft guidance documents for completing the international SIP). For example, the SMC guidance highlights the Association of the British Pharmaceutical Industry (ABPI) code of practice, which requires information provided to the public as part of an HTA to be accurate, not misleading, and non-promotional in nature; it must be factual and presented in a balanced way (11). Compliance with local requirements and the support of industry trade associations in different countries was highlighted as being critical to the implementation of the SIP. It is important to note that, although the development of the International SIP has been driven by the HTAi PCIG with input provided by industry partners as part of this process, local implementation of the SIP will still require support from the relevant industry body.
In this project, the timing of patient involvement during the HTA process was highlighted as a key consideration for implementation of the SIP. Although many HTA bodies already share information with patient groups, this may be at a late stage. For example, in NICE assessments, the pharmaceutical company and patient group submit materials at the same time. This highlights the necessity for the SIP document to be introduced and drafted at the initiation of the HTA process, which would require resources and be a challenge for implementation of a standardized international SIP. Similarly, although ownership and responsibility for the document lies with the submitting pharmaceutical company, HTA bodies will need to choose whether to review the completed SIP for objectivity before sharing with patient groups, in order to give reassurance that appropriate information has been provided. This would again require early involvement of the SIP in the HTA submission process.
Discussions with patient groups and HTA bodies in different countries also examined how an international SIP template may be received in each country, resulting in positive feedback and suggested changes. HTA processes vary considerably depending on location, culture, clinical practices, and commercial arrangements. As such, it is anticipated that individual bodies will need to adapt the SIP template, removing sections that are not applicable to them and potentially developing their own additional guidance for use. The template should, therefore, be identifiable as something that is flexible and can be adapted locally, while recognizing the value of consistency.
Limitations
There are a number of limitations in relation to the project findings. The response rate to the patient survey was relatively low, and assessment of the feasibility of the SIP approach was based on the feedback from a limited number of HTA agencies. In addition, patient groups were sampled due to previous participation in the HTA process; however, this may mean that the perspectives and needs of those who are less familiar with HTA processes may not be fully accounted for in the current draft SIP template. Similarly, the semistructured interviews were undertaken only with representatives from high-income countries, and it is not known what challenges may be faced by HTA bodies in other environments. With the availability of the draft international SIP template and potential sharing of locally adapted resources or evaluations, it is anticipated that this will enable other HTA bodies and patient groups to provide feedback and facilitate further examination of these issues. This is likely to include a larger sample of patient groups as well other users, such as individual patients and carers.
It should also be noted that various lay summary formats for general communication of scientific information to patients exist. Although use of the SMC SIP as the model for development of a patient-friendly summary could be viewed as a potential limitation of this project, this format has previously been shown to be received positively by patient groups and pharmaceutical companies (Reference Single, Facey, Livingstone and Silva6).
Future Developments
There are several ongoing and planned developments. Awareness of the template will continue to be broadened to bring it to the attention of industry trade associations and HTA bodies in additional countries to explore the feasibility of the template and/or undertake pilots. Additional engagement with patient groups is also planned to spread awareness and further explore the benefits of the SIP to involvement in HTA deliberations and decision making.
It is acknowledged that some local patient groups may struggle to review materials in English. Although the work during the development of the international SIP was conducted in English, it is intended that the final materials will be translated into other languages. It is anticipated that local patient groups will be involved in translation work to ensure that the intended meaning of the information is retained and is understandable.
The international SIP template developed here focuses on evaluations of medicinal products/pharmaceuticals. It is anticipated that a follow-on project may be needed to adapt the template for use with medical devices and diagnostics.
Conclusions
The International SIP template has been developed to support patient and patient group input into HTA deliberations. This approach supports the principle of inclusivity and may also have benefits in terms of supporting transparency and improving health literacy. Qualitative data from stakeholders with experience of using the SMC SIP template indicated that SIPs can be helpful to patient groups when introduced and at the start of the HTA process. The sequencing of when to provide the SIP template to patient groups is a critical issue for HTA bodies to consider as more meaningful involvement from patient groups is likely to occur with earlier provision of information. For both HTA bodies and pharmaceutical companies, there may also be issues to consider around resourcing, particularly in terms of writing or reviewing the summary for a patient audience. The potential benefit of the international template is that sections of the template can be shared locally to reduce some of this burden.
Local adaptation of the template and resources will also allow for the refinement of the international SIP template. Feedback from the SMC experience including comments from pharmaceutical companies and patient groups familiar with the SMC SIP along with observations from patient groups less familiar with the SIP approach led to the development of a version for implementation. However, the HTAi PCIG project team recognizes the importance of HTA bodies and pharmaceutical companies using the template and making iterative changes based on evaluations done locally. These improvements are ultimately expected to help provide better-informed HTA decision making.
Acknowledgments and sources of funding
No specific funding was provided for this project. Under direction of the authors, medical writing and strategic input was provided by agencies Ashfield Healthcare Communications, part of UDG Healthcare, and Oxford PharmaGenesis. This was during the research and development phases of the international SIP template and the supporting guidance documentation. Funding for this support was provided by Novartis. All authors are members of the Health Technology Assessment international (HTAi) Interest Group for Patient and Citizen Involvement in Health Technology Assessment (PCIG).
Conflicts of Interest
NC, HL, JD, LT, HM, MC, SW, EO, MJVE, FW, BL: Nothing to disclose.
KM: Reports grants from Novartis, grants from Bristol Myers Squibb, grants from Celgene, grants from Amgen, grants from Takeda, grants from GSK, grants from Sanofi, grants from Janssen, grants from Mundipharm, grants from Karyopharm, grants from Pfizer, grants from Roche, outside the submitted work.
EO: Reports grants from National Pharmaceutical Council, other from Patient-Centered Outcomes Research Institute, outside the submitted work; the National Health Council is a membership organization that receives dues and sponsorships from a variety of funders, both members and sponsors. The complete lists of members and sponsors can be found at http://www.nationalhealthcouncil.org.
Open access
Introduction
Understanding the perspectives of patients and their families and carers is becoming widely recognized as a critical component in any Health Technology Assessment (HTA) for reimbursement purposes (1–Reference Weeks, Polisena, Scott, Holtorf, Staniszewska and Facey3). Patients, families, and carers can provide unique information about what it is like to live with a condition and give a real-life view of the potential impact of a new medicine (Reference Perfetto, Oehrlein, Boutin, Reid and Gascho4). Patient groups can help provide this information through their engagement with the HTA process, which is illustrative of inclusivity, a core principle of effective deliberative processes in HTA (Reference Bond, Stiffell and Ollendorf5). Inclusivity involves creating an environment where stakeholders can meaningfully contribute to the HTA decision-making process, for example by sharing information and/or providing this information in plain language. Often, the content of an HTA submission is inaccessible or quite technical. As a result, patients or patient groups may find it difficult to comment on the proposed use of a treatment or discuss trade-offs of the benefits and harms of a treatment. HTA bodies, however, often ask patients or patient groups these questions to inform deliberations in addition to questions about patients’ experience of living with a condition (Reference Single, Facey, Livingstone and Silva6–Reference Sampietro-Colom and Thomas8). It is, therefore, important that relevant patient audiences have an informed and appropriate understanding of the treatment or technology under review in order to optimize their input into the HTA process.
Many HTA bodies emphasize the involvement of patient groups and have developed working practices to solicit input to inform these deliberative processes (Reference Facey, Hansen and Single9). These include the use of consultation surveys and consultation documents. These also include the initiative of the Scottish Medicines Consortium (SMC) to develop a template for pharmaceutical companies to complete that could be provided to patient groups as a plain language summary of the medicine under appraisal (Reference Single, Macpherson, Fearns, Dickson, Facey, Facey, Ploug Hansen and Single10). In 2017, the SMC introduced an update to this template in the form of the Summary Information for Patient Groups (SIP) (11). This later became a mandatory part of HTA submissions to the SMC. However, most other HTA bodies have yet to follow the example of the SMC, and patient input to the HTA process is often in the absence of or with limited information about the intervention (drug, diagnostic, or device) under evaluation. Pharmaceutical companies making an HTA submission to the SMC are asked to provide answers to seven questions, covering the following: the use of the medicine, management of the condition in Scotland, how the medicine works, its effectiveness and safety, and the impact of the medicine on quality of life for patients and carers. The SIP is made available by the SMC as a resource to support the SMC Patient Group Submission process (12).
The Health Technology Assessment international (HTAi) Interest Group for Patient and Citizen Involvement in HTA (PCIG) project was initiated in response to recognition of the value of providing patient groups with clear information, and to help provide tools to support and encourage robust and meaningful involvement of the patient perspective in HTA deliberations. Two specific objectives of the project were (1) to collect qualitative data to assess the experience of using the SMC SIP and (2) to develop an international SIP template based on these findings.
Here, we present the results of the qualitative evaluation of the SMC SIP from pharmaceutical company representatives, HTA bodies, and patient groups. Data collection methods consisted of the following: a survey of pharmaceutical representatives on their experience of using the SMC SIP; a survey of patient groups on their experience of receiving completed SIP documents as part of HTA applications in Scotland; semistructured interviews with representatives of HTA bodies; and discussions with patient groups in the United States, Australia, and Europe to gather feedback on the SIP approach more generally. These activities were conducted by the HTAi PCIG project team. Formal research ethics approval was not sought for these activities as this was considered a quality improvement project. These qualitative data were used to inform the development of an international SIP template and associated guidance documents. Although various “lay summary” formats exist, the SMC SIP was chosen as the basis for the International SIP template as it is already in use in Scottish HTAs and is known by pharmaceutical companies that would need to accept this as an additional part of their submission process. Furthermore, patient groups have found the format to be helpful (Reference Single, Facey, Livingstone and Silva6). Further information regarding the SMC SIP can be found under Section 8 of the New Product Assessment Form at: https://www.scottishmedicines.org.uk/making-a-submission/ and in the Guidance to Manufacturers document at: https://www.scottishmedicines.org.uk/media/2771/guidance-on-summary-information-for-patient-groups.pdf.
Methods
Surveys of Pharmaceutical Company Representatives and Patient Groups
Two online surveys were designed by members of the SMC public involvement team in consultation with the HTAi PCIG project team to obtain feedback from pharmaceutical companies and patient groups on their experience of either completing or receiving an SIP relating to medicines submitted to the SMC for appraisal between October 2018 and November 2019. The two e-surveys were developed using SmartSurvey build version 5.5.0.1809 and consisted of a mix of open and closed questions (e-surveys available in the Supplementary Material).
The e-survey of pharmaceutical company representatives consisted of thirteen questions, and respondents were advised that it would take approximately 10 minutes to complete. Closed questions on simple agreement scales asked respondents to assess: whether pharmaceutical companies value SIP forms; whether the SMC SIP form resulted in better-informed patient input; whether it was easy to complete; the sufficiency of the guidance provided for content and writing style. Respondents were also asked whether and how they assessed the readability of the SIP form. Open questions asked respondents to expand on these answers and suggest any improvements to the SIP.
For the e-survey of patient group representatives on the experience of receiving the SMC SIP, closed questions asked respondents to assess: whether they read the form and found it helpful; how helpful they found each section of the SMC SIP; whether the information was at the right level of detail; ease of completion and clarity of language; and whether the information in the form was credible. Open questions asked respondents to expand on these answers. The patient e-survey consisted of twelve questions, with respondents being advised that it should take approximately 10 minutes to complete.
A link to each e-survey was disseminated by the SMC by e-mail in November 2019 to pharmaceutical company representatives (n = 32) and patient group representatives (n = 55). A 3-week period was allowed for completion of the surveys. Responses were anonymous. Descriptive statistics were used to analyze responses to the closed questions. Responses to open questions were used to understand the reasoning behind answers and to explore themes relating to the user experience.
Semistructured Interviews with HTA Body Representatives
Experience of implementing the SIP from an HTA agency's perspective was collected through semistructured interviews with staff from the SMC who had been involved with the SIP process. The semistructured interview guide was developed by the HTAi PCIG project team and covered the rationale for introducing the SIP, perceived or expected benefits of the SIP, and barriers or challenges associated with the implementation of the SIP.
Further interviews were conducted with senior representatives and/or staff at the National Institute for Health and Care Excellence (NICE) in England, the Pharmaceutical Benefits Advisory Committee (PBAC) Secretariat in Australia, and Aragon Health Sciences Institute (IACS) in Spain. These interviews focused on understanding the feasibility of implementing an SIP in the local context and potential barriers and challenges to implementation.
Discussions with Patient Groups
Patient/consumer group discussions were held in Europe, Australia, and the USA to explore the value of the SIP approach for patient groups and to examine what information would be considered useful for these groups to assist with HTA participation.
Discussions with patient groups in Australia and the USA were connected to formal patient group meetings that were already planned. Two example-completed SIPs were shared in advance of these meetings to facilitate discussions. Patient representatives provided feedback regarding the content of the SIP template (e.g., what was helpful, what could be added, etc.), language (e.g., tone, perceived bias, readability, etc.), and other comments regarding the use of the SIP in their respective countries. For the U.S. meeting, two SMC staff members joined virtually to introduce the SIP template and answer questions about how the SMC uses the SIP as part of an HTA assessment. European groups were contacted and provided informal feedback via e-mail or interviews. Prompting questions for discussion included whether SIP forms provide useful information, whether any key information was missing from example SIPs, and how SIPs might be improved. Sampling for all groups was conducted on a convenience basis, based on existing networks to provide input from a variety of patient groups from three different regions of the world.
Results
Evaluation of the SMC SIP Template and SIP Approach
Survey of Pharmaceutical Company Representatives
Out of thirty-two pharmaceutical company representatives who received the survey link from the SMC, twenty-two responded to the questionnaire (response rate: 69%). Most industry respondents (18/22, 82%) agreed that completing the SIP form was a worthwhile investment of their time, with four respondents (18%) indicating that they partially agreed. When asked if the SIP form resulted in more informed patient input to the SMC's appraisal process, 62 percent (14/22) agreed or partially agreed, 27 percent (6/22) stated that they did not know, and some explaining in free text that they had not received feedback from patient groups and that this would be beneficial in the future. The respondents who did not agree (2/22, 9%) gave no explanation for their disagreement.
Almost all respondents (21/22, 95%) agreed that the need for the SIP was clearly explained. Overall, 73 percent (16/22) of respondents reported that the SIP form was “very easy” (3/22, 14%) or “easy” (13/22, 59%) to complete. The remainder (6/22, 27%) reported that the form was “Neither easy nor difficult.” Although none of the respondents reported that it was difficult to complete the SIP form, free-text responses highlighted that writing in plain English can be challenging (Table 1).
Table 1. Direct quotes from e-survey of pharmaceutical company representatives’ perspectives on the SMC SIP form
SIP, Summary of Information for Patients; SMC, Scottish Medicines Consortium.
Respondents agreed (19/22, 86%) or partially agreed (3/22, 14%) that there was sufficient guidance on how to complete the SIP in terms of content, and agreed (14/22, 64%) or partially agreed (8/22, 36%) that there was sufficient guidance in terms of writing style/level. Suggestions for additional guidance included provision of best practice example SIP forms, advice on producing lay documents, and a glossary of terms. Most respondents (16/22, 73%) reported that there was no “plain English check” of SIP documents when they were drafted (e.g., with a readability assessment such as the Flesch reading ease test (Reference Spadaro, Robinson and Smith13)).
When asked how the SIP form could be improved, the majority of respondents reported that the content of the SIP form was appropriate (19/22, 86%). Two respondents (14%) reported that additional information was required and suggested including more information on cost-effectiveness and statistics to help the patient representatives interpret the data.
Survey of Patient Group Representatives
Out of fifty-five patient groups contacted by the SMC, seventeen survey responses were received (response rate: 31%). Of these, most (15/17, 88%) reported that they had read the information in the SIP form. The respondents who did not read the SIP form (2/17, 12%) explained that they prefer to conduct their own research and refer to independent sources as they believe that the information in the SIP form would bias their submission to the HTA. Of the fifteen who reported reading the SIP, ten (67%) reported finding the document “helpful” and five (33%) reported it to be “very helpful.” Open question responses suggested that the SIP form was used in different ways, including: to improve understanding of the technical aspects of a new medicine; to complement existing knowledge; to gain insight into the company viewpoint; and to inform communication with patients/families when eliciting their perspectives.
Each section of the SIP form was considered “very helpful,” “helpful,” or “slightly helpful” by all respondents; no sections were identified as “not at all helpful.” When asked which section was the most helpful, individuals’ free-text responses were highly varied (Table 2).
Table 2. Direct quotes from e-survey of patient groups’ perspectives on the SMC SIP form
HCP, Healthcare Professional; HTA, Health Technology Assessment; SIP, Summary of Information for Patients; SMC, Scottish Medicines Consortium.
In general, patient groups felt that the format of the SIP form was easy to use and that the information provided was at an appropriate level of detail, although opinions were mixed regarding understandability. Although many patient group respondents agreed that the terminology used in the SIP form was well explained, others felt there was still too much technical language and abbreviations. One respondent noted that too much scientific language could potentially put people off reading the SIP.
In total, seven respondents made suggestions for additional content for the SIP form, including: information on where the new medicine belongs in the existing treatment pathway; details of when the medicine would be made available; and information on the side effects of the new medicine compared with existing treatment(s).
Semistructured Interviews with HTA Body Representatives
Representatives of the SMC generally reported positive feedback from implementing the SIP and felt that it had improved patient group involvement in the HTA process, with the number of submissions with input from patient groups increasing along with an overall improvement in the quality of the patient group submissions.
Prior to the development of the SIP, the SMC asked patient groups to complete submission forms, but did not provide information about the medicine being assessed. Representatives of the SMC reported that some patient groups found this a challenge and that the development of the SMC SIP had helped overcome this issue. Furthermore, the SIP had led to increased satisfaction among patient groups and a greater feeling that they were being supported by the SMC.
It was noted that potential concerns with the approach had been that patient groups could infer that an SIP document may contain biased information because it had been drafted by pharmaceutical companies, and that patient groups would replicate information provided rather than using the information as a basis to inform their own submission as to what matters most to patients. It was reported that this had occurred only in a small number of cases and that where it had occurred, the SMC public involvement team, as part of their review process of the SIP and patient submissions, had asked for those documents to be revised.
Interviews with representatives from HTA bodies on the feasibility of the SIP identified several aspects to be considered when implementing it. These considerations included the importance of establishing agreement from the relevant industry body, understanding where the SIP would fit into the sequencing of the HTA process, who would be responsible for the content, and what resources would be needed to implement the SIP locally (Table 3).
Table 3. Aspects for HTA agencies to consider when implementing the SIP
HTA, Health Technology Assessment; HTAi PCIG, Health Technology Assessment International Interest Group for Patient and Citizen Involvement; SIP, Summary of Information for Patients.
Feedback from Patient Group Sessions
Patient group feedback was provided by representatives from 6 European patient groups, representatives from 18 U.S. patient groups, and representatives from 12 Australian patient groups. Table 4 summarizes feedback provided by the patient groups. Overall, patient groups liked that the SIP documents were concise, used a question and answer approach, and were formatted with bullet points and/or tables.
Table 4. Summary of feedback from patient groups on the summary of information for patients template on potential information and layout
Some patient representatives, however, commented that the documents were still quite dense and noted that some patient groups would not have the level of expertise and/or health literacy required to interpret these documents. To assist with understanding, diagrams and infographics could be used, and readability assessments could be undertaken to check complexity in addition to a patient reviewer prior to it being distributed.
Some patient groups also suggested that an executive summary was needed at the beginning of the document to help highlight key points prior to reading and that a glossary of terms would be useful. Patient groups also wanted to know how the document would be distributed, whether it would be available to individual patients (as distinct from patient groups), and what role the HTA body would have in reviewing the content.
Development of an International SIP Template and Preparation for Local Implementation
The findings from the project-related activities were used to draft an international SIP template consisting of four key sections: (1) disease background information; (2) information on the new treatment and its potential therapeutic benefits; (3) information on the economic value of the new treatment; (4) further resources that may be of interest to the reader. Compared with the SMC SIP template, the proposed international version includes key changes that can be summarized as follows: inclusion of an economic value section, a patient-based evidence section, a glossary, and generalization beyond Scotland. The qualitative findings from the survey and patient group sessions were also used to refine the language used in the template and to develop guidance documents to support industry, HTA bodies, and patient groups in the completion, assessment, and use of the SIP. These documents, along with the International SIP template (Version 1), can be found on the HTAI Web site (https://htai.org/interest-groups/pcig/projects/current-projects/).
Discussion
The importance of the patient perspective in HTA is increasingly appreciated, and the growing participation of patient groups in HTA in some countries has shown positive effects. Nevertheless, there remains a need for improvements and consistency in approaches used, taking into account both common (e.g., time and resource constraints, “real-world” practicalities, etc.) and country-specific challenges (Reference Hailey, Facey, Ploug Hansen and Single14;Reference Wong-Rieger, Facey, Ploug Hansen and Single15). The international SIP template is intended to become a consistent information resource available to support patient and patient group input into HTA, which can be shared by HTA bodies and completed by the industry as part of their submissions.
HTA submission documents are frequently written using technical terms that may not be understandable to lay audiences. Although the current SMC SIP received positive feedback, a few of the patient groups involved in this project reviewed past SMC SIPs and considered the language overly technical. Provision of a suitable SIP template may help standardize language at a more appropriate level for patient groups. This also raises the need for industry to develop the capabilities to write for a lay audience and/or for HTA bodies to provide training and support in this area. The qualitative results collected here suggest that further guidance on the use of plain language may help (such as more detailed guidance documents, provision of a glossary, and the use of readability assessments and a lay/patient reader).
Regardless of these initiatives, some patient groups may choose not to use an SIP. Our findings suggest that this may be because some groups are comfortable undertaking their own research of the medical literature, and/or are familiar with the medicine under evaluation, or have concerns around the credibility and potential bias associated with information being provided by pharmaceutical companies. Given that patient advocacy groups are highly heterogeneous, this is expected. There are significant differences in background, access to resources, expertise, and experience with HTA processes among patient groups, as well as differing attitudes toward pharmaceutical companies. HTA bodies are best placed to undertake activities that support impartiality, transparency, and inclusivity. In implementing the international SIP template, this may involve providing regular guidance, training, and feedback to all stakeholders, as well as implementing a formal review process of the SIP prior to distribution to patient groups.
One key challenge for the creation and standardization of an international SIP template is that the provision of information to patient groups by pharmaceutical companies will be subject to different regulations in different countries (this is emphasized in draft guidance documents for completing the international SIP). For example, the SMC guidance highlights the Association of the British Pharmaceutical Industry (ABPI) code of practice, which requires information provided to the public as part of an HTA to be accurate, not misleading, and non-promotional in nature; it must be factual and presented in a balanced way (11). Compliance with local requirements and the support of industry trade associations in different countries was highlighted as being critical to the implementation of the SIP. It is important to note that, although the development of the International SIP has been driven by the HTAi PCIG with input provided by industry partners as part of this process, local implementation of the SIP will still require support from the relevant industry body.
In this project, the timing of patient involvement during the HTA process was highlighted as a key consideration for implementation of the SIP. Although many HTA bodies already share information with patient groups, this may be at a late stage. For example, in NICE assessments, the pharmaceutical company and patient group submit materials at the same time. This highlights the necessity for the SIP document to be introduced and drafted at the initiation of the HTA process, which would require resources and be a challenge for implementation of a standardized international SIP. Similarly, although ownership and responsibility for the document lies with the submitting pharmaceutical company, HTA bodies will need to choose whether to review the completed SIP for objectivity before sharing with patient groups, in order to give reassurance that appropriate information has been provided. This would again require early involvement of the SIP in the HTA submission process.
Discussions with patient groups and HTA bodies in different countries also examined how an international SIP template may be received in each country, resulting in positive feedback and suggested changes. HTA processes vary considerably depending on location, culture, clinical practices, and commercial arrangements. As such, it is anticipated that individual bodies will need to adapt the SIP template, removing sections that are not applicable to them and potentially developing their own additional guidance for use. The template should, therefore, be identifiable as something that is flexible and can be adapted locally, while recognizing the value of consistency.
Limitations
There are a number of limitations in relation to the project findings. The response rate to the patient survey was relatively low, and assessment of the feasibility of the SIP approach was based on the feedback from a limited number of HTA agencies. In addition, patient groups were sampled due to previous participation in the HTA process; however, this may mean that the perspectives and needs of those who are less familiar with HTA processes may not be fully accounted for in the current draft SIP template. Similarly, the semistructured interviews were undertaken only with representatives from high-income countries, and it is not known what challenges may be faced by HTA bodies in other environments. With the availability of the draft international SIP template and potential sharing of locally adapted resources or evaluations, it is anticipated that this will enable other HTA bodies and patient groups to provide feedback and facilitate further examination of these issues. This is likely to include a larger sample of patient groups as well other users, such as individual patients and carers.
It should also be noted that various lay summary formats for general communication of scientific information to patients exist. Although use of the SMC SIP as the model for development of a patient-friendly summary could be viewed as a potential limitation of this project, this format has previously been shown to be received positively by patient groups and pharmaceutical companies (Reference Single, Facey, Livingstone and Silva6).
Future Developments
There are several ongoing and planned developments. Awareness of the template will continue to be broadened to bring it to the attention of industry trade associations and HTA bodies in additional countries to explore the feasibility of the template and/or undertake pilots. Additional engagement with patient groups is also planned to spread awareness and further explore the benefits of the SIP to involvement in HTA deliberations and decision making.
It is acknowledged that some local patient groups may struggle to review materials in English. Although the work during the development of the international SIP was conducted in English, it is intended that the final materials will be translated into other languages. It is anticipated that local patient groups will be involved in translation work to ensure that the intended meaning of the information is retained and is understandable.
The international SIP template developed here focuses on evaluations of medicinal products/pharmaceuticals. It is anticipated that a follow-on project may be needed to adapt the template for use with medical devices and diagnostics.
Conclusions
The International SIP template has been developed to support patient and patient group input into HTA deliberations. This approach supports the principle of inclusivity and may also have benefits in terms of supporting transparency and improving health literacy. Qualitative data from stakeholders with experience of using the SMC SIP template indicated that SIPs can be helpful to patient groups when introduced and at the start of the HTA process. The sequencing of when to provide the SIP template to patient groups is a critical issue for HTA bodies to consider as more meaningful involvement from patient groups is likely to occur with earlier provision of information. For both HTA bodies and pharmaceutical companies, there may also be issues to consider around resourcing, particularly in terms of writing or reviewing the summary for a patient audience. The potential benefit of the international template is that sections of the template can be shared locally to reduce some of this burden.
Local adaptation of the template and resources will also allow for the refinement of the international SIP template. Feedback from the SMC experience including comments from pharmaceutical companies and patient groups familiar with the SMC SIP along with observations from patient groups less familiar with the SIP approach led to the development of a version for implementation. However, the HTAi PCIG project team recognizes the importance of HTA bodies and pharmaceutical companies using the template and making iterative changes based on evaluations done locally. These improvements are ultimately expected to help provide better-informed HTA decision making.
Supplementary material
The supplementary material for this article can be found at https://doi.org/10.1017/S0266462321000167.
Acknowledgments and sources of funding
No specific funding was provided for this project. Under direction of the authors, medical writing and strategic input was provided by agencies Ashfield Healthcare Communications, part of UDG Healthcare, and Oxford PharmaGenesis. This was during the research and development phases of the international SIP template and the supporting guidance documentation. Funding for this support was provided by Novartis. All authors are members of the Health Technology Assessment international (HTAi) Interest Group for Patient and Citizen Involvement in Health Technology Assessment (PCIG).
Conflicts of Interest
NC, HL, JD, LT, HM, MC, SW, EO, MJVE, FW, BL: Nothing to disclose.
KM: Reports grants from Novartis, grants from Bristol Myers Squibb, grants from Celgene, grants from Amgen, grants from Takeda, grants from GSK, grants from Sanofi, grants from Janssen, grants from Mundipharm, grants from Karyopharm, grants from Pfizer, grants from Roche, outside the submitted work.
EO: Reports grants from National Pharmaceutical Council, other from Patient-Centered Outcomes Research Institute, outside the submitted work; the National Health Council is a membership organization that receives dues and sponsorships from a variety of funders, both members and sponsors. The complete lists of members and sponsors can be found at http://www.nationalhealthcouncil.org.