Horizon scanning is a systematic process that serves as an early warning system to inform decision makers about possible future opportunities and threats. Healthcare horizon scanning identifies novel technologies, innovations, or new uses of existing technologies (e.g., drugs, devices, procedures, diagnostics, and care processes) and trends with potential for future disruptions (i.e., positive or negative major changes). Disruptions could affect care access, delivery, setting, and costs; current diagnostic or treatment models; health disparities; healthcare infrastructure; patient health outcomes; and public health (Reference Sun and Schoelles1–Reference Hulshizer, Lynch, De Lurio, Erinoff, Robertson and Tipton3).
Healthcare horizon scanning has informed a variety of strategic planning activities. Public and private entities globally have long used information from formal or informal healthcare horizon scanning programs to inform commercial planning decisions, health services and clinical research prioritization, financial or operational planning, controlled diffusion of technologies, and information dissemination to policy makers, purchasers, and healthcare providers. Health insurance companies and government payers have used horizon scanning to inform coverage decisions. Also, some organizations, such as ECRI and EuroScan, have used it to inform decisions about health technology assessment (HTA) activities; indeed, horizon scanning marks the first step in evidence assessment (Reference Hulshizer, Lynch, De Lurio, Erinoff, Robertson and Tipton3).
From 2010 through 2015, the U.S. Agency for Healthcare Research and Quality (AHRQ) engaged ECRI to design, develop, and implement the first U.S. national Healthcare Horizon Scanning System (HHSS). It was primarily intended to inform comparative-effectiveness research investments made through AHRQ's Effective Health Care Program (4). The system's scope was broad—spanning therapeutics, diagnostics, and programs in fourteen AHRQ-defined priority areas (e.g., cancer and obesity) and a cross-cutting category (e.g., topics spanning multiple conditions). The process obtained inputs from clinical and health systems stakeholders, but not from patient stakeholders, to inform decisions about which interventions had the highest potential to address unmet needs (Reference Hulshizer, Lynch, De Lurio, Erinoff, Robertson and Tipton3). The stakeholder recruitment process was similar to the Patient-Centered Outcomes Research Institute (PCORI) Health Care Horizon Scanning System (HCHSS) process (Reference Hulshizer, Lynch, De Lurio, Erinoff, Robertson and Tipton3). ECRI maintains a network of more than three hundred medical and scientific experts in more than twenty-five specialties.
In late 2018, PCORI engaged ECRI to implement and operate the PCORI HCHSS (https://www.pcori.org/research-results/evidence-synthesis/pcori-health-care-horizon-scanning-system) to better inform research investments (5). It builds on concepts and approaches from the AHRQ system. Although the PCORI HCHSS protocol is similar, the scope of the HCHSS is more focused, with one notable exception—healthcare trends. The PCORI HCHSS scans for and monitors interventions in five priority areas: Alzheimer's disease and other dementias, cancer, cardiovascular disease, mental and behavioral health conditions, and rare diseases. Included interventions are anticipated to be clinically available in the United States outside the research environment within 3 years and are monitored up to 1 year after initial clinical availability (e.g., regulatory approval) (Reference Hulshizer, Lynch, De Lurio, Erinoff, Robertson and Tipton3). The system helps PCORI decide how to invest research funds. The public and research community can use the published information for research and decision-making.
The HCHSS also identifies high-level disruptive trends that occur across or within clinical areas that may create a paradigm shift in health care. PCORI does not limit the trends’ scope to the five priority areas (Reference Hulshizer, Lynch, De Lurio, Erinoff, Robertson and Tipton3). For example, the continued and rapid development of expert artificial intelligence systems in health care cuts across all clinical areas and increasingly transforms screening, diagnosis, and treatment (Reference Hulshizer, Carlson, Cuevas, Druga, Lynch and Mehta6).
A HCHSS hallmark is the inclusion of patient and caregiver perspectives in addition to clinical, health systems, and research perspectives. These added perspectives introduce more diverse thinking about an intervention's disruption potential (Reference Hulshizer, Lynch, De Lurio, Erinoff, Robertson and Tipton3), such as how an intervention might affect health outcomes, adherence, or care processes. Furthermore, this stakeholder group supports PCORI's initiatives to engage patients in research. This paper aims to describe our early experience and results from engaging patients and caregivers in the HCHSS process to help determine which new healthcare interventions have the highest disruption potential.
Methods
Horizon Scanning Process
The PCORI HCHSS process involves systematic, ongoing, scanning of hundreds of key information sources, from news aggregators like PRNewswire to clinical trial registries (ClinicalTrials.gov), trade publications, and peer-reviewed journals to identify topics (i.e., interventions) and trends likely to disrupt health care within the next 3 years. Analysts review potential topics and trends and, if interventions or trends meet the HCHSS protocol's inclusion criteria (Reference Hulshizer, Lynch, De Lurio, Erinoff, Robertson and Tipton3), enter them into a database for nomination. At topic nomination meetings, analysts present each topic or trend to the HCHSS team. After brief discussion, the team uses a blind voting tool to include or exclude the topic or trend (based on protocol criteria). All included topics and trends are reported in a publicly available quarterly Status Report (Reference Hulshizer, Carlson, Cuevas, Druga, Hudson and Lynch7) that briefly describes each topic. Topics with late-phase clinical data are further developed into more detailed reports—called topic profiles—compiled from the results of focused searches of published clinical data. For each topic profile, comments are obtained from five to nine stakeholders. The goal is to include at least one patient representative (e.g., patient or advocate) or caregiver in each stakeholder group for each topic profile.
The selected stakeholders are familiar with the topic profile's clinical area, the disease or condition, or the health system generally so they can provide perspectives based on their knowledge and experience. The stakeholders engaged vary by topic profile. Each stakeholder reads the topic profile and completes a brief comment form, which elicits ratings on the intervention's disruption potential on a scale of one (not disruptive) to four (highly disruptive). A brief written rationale is required to support each rating.
Twice yearly, PCORI HCHSS analysts review stakeholder ratings and comments received in the previous 15 months for each topic profile to nominate topics deemed to have the highest disruption potential for inclusion in a High Potential Disruption Report (HPDR) (Reference Hulshizer, Lynch, De Lurio, Erinoff, Robertson and Tipton3). See Figure 1 for a process overview.
Figure 1. Presents an overview of the ongoing HCHSS process. Step one involves broad scanning by information specialists to identify leads for potential topics. In step two, analysts review leads to identify topics that meet disruption criteria. In step three, topics are nominated, discussed, and voted on by the team for inclusion. Step four involves topic development followed by step five when topics with late-phase data are sent for stakeholder comment. The last step in the process, step six, is the development of the HPDR.
Patient Stakeholder Recruitment Process
A Patient Engagement Specialist (PES) with a background in community outreach and program planning recruits individual patient and caregiver stakeholders from credible community sources, such as national advocacy organizations or healthcare consortiums (e.g., Consumers United for Evidence-based Healthcare). Patient stakeholders include people who have lived experience with a disease or condition, patient representatives, and patient advocates; we use the term “patients” to describe all three types. The PES searches the Internet for organizations relevant to the five priority areas and identifies a point of contact (e.g., outreach coordinator). Information about the identified organizations, such as the mission and contact detail, is tracked using Microsoft Office Excel 2016. The PES e-mails the contact, briefly describing the HCHSS and importance of patients and caregivers as commenters, and asks for support in identifying potential participants. The PES also extends an invitation to discuss the project further during a conference call. Our recruitment methods are informed by PCORI's Engagement Rubric (8). The system protocol provides further details about our methods (Reference Hulshizer, Lynch, De Lurio, Erinoff, Robertson and Tipton3).
Once a patient or caregiver is identified, the Stakeholder Engagement Coordinator (SEC) e-mails a welcome message describing the HCHSS and commenter responsibilities and requests details about the stakeholder's lived experience and role. Next, the PES e-mails the stakeholder a 10-minute audiovisual presentation describing project goals, the importance of engaging patient stakeholders in the process, key terminology, and how comments are used. A sample report and comment form are also provided.
When a topic profile suiting their experience is ready for review, stakeholders receive an e-mail query from the SEC that they may accept or decline. Acceptance automatically sends the five- to seven-page topic profile and comment form link to the stakeholders' inbox. They have 5 business days to submit their review electronically. Stakeholders are asked to disclose potential conflicts of interest. Conflicts do not limit participation. Instead, we seek to balance their views with inputs from other neutral parties, including ECRI's own experts. The SEC tracks and logs comment form submissions into a database (where they are stored for analyst review), sends reminders to complete assigned reviews, and may extend deadlines as needed. External stakeholders (non-ECRI employees) complete a U.S. Internal Revenue Service W-9 tax form to receive an honorarium of $250 for each report reviewed with comment form completed. The SEC who manages the review process is a medical copyeditor who draws on experience obtaining reviews, a typical copyeditor task, from the AHRQ HHSS experience.
How Stakeholder Comments Are Used
The stakeholders’ comments and ratings for each topic profile, including those from patients and caregivers, inform analysts about which topics to recommend as having high disruptive potential (Reference Hulshizer, Lynch, De Lurio, Erinoff, Robertson and Tipton3). Throughout this process, patients and caregivers have provided unique perspectives (Table 1).
Table 1. Effect of patient stakeholder comments on analyst inclusion recommendations for the HPDR
Note: This table shows patient stakeholder perspectives with their impact on inclusion recommendations for the May 2019 and November 2019 editions of the HPDR. The five column headings from left to right are: topic, stakeholder type, comment summary, analyst assessment of comments, and effect on analyst inclusion recommendation for report. The topics are arranged vertically by clinical condition with the following headings: Alzheimer's disease topic (row two), cancer topics (row four), cardiovascular disease topics (row eight), and rare disease topics (row twelve).
Hbs/S, hemoglobin S; HNSCC, head and neck squamous cell carcinoma; HSCT, hematopoietic stem cell transplantation; SCD, sickle cell disease; TDT, transfusion-dependent β-thalassemia; TNBC, triple-negative breast cancer.
Analysts review stakeholders’ numeric ratings and comments for each topic profile and note whether strong consensus or variation exists, whether the disruption potential crosses multiple parameters (e.g., patient outcomes and delivery of care), and the overall projected disruption magnitude.
Analysts may identify topics as potentially highly disruptive when similar substantive comments and scores are received across many parameters or by most or all commenters. Conversely, comments and scores may indicate a substantial anticipated effect, but only in one or a small number of parameters or by a small number of stakeholders. Although no particular stakeholder's comments receive more weight at the outset, analysts may weigh more heavily comments from those who demonstrate considerable experience and knowledge in a clinical area. For example, comments from stakeholders in direct contact with patients (e.g., clinicians) or directly affected by the disease or condition (e.g., caregivers) may be deemed more pertinent.
Patient Stakeholder Experience Survey
ECRI collects and evaluates patients’ and caregivers’ views of this process by administering a participation experience survey. Survey administration began on 4 June 2019 using Checkbox survey software. The survey includes seven questions with a four-point Likert-type scale (agree [four points], somewhat agree, somewhat disagree, and disagree [one point]). We chose a four-point scale for similarity to the scale used on the topic profile rating and comment form and to avoid respondents’ tendency to choose the mid point. Question seven includes a free-text box for respondents to explain their response. Question eight allows respondents to enter additional comments in a free-text box. Respondents are also asked which stakeholder group(s) they represent (e.g., patient and caregiver).
After a participant's first review is received, the survey is e-mailed to the participant (Supplementary File 1). One week later, a reminder e-mail is sent if a completed survey has not been received. Checkbox captures the survey results. The PES downloads them into Microsoft Excel and reviews them, and then a senior member of the project leadership team reviews them. Our team then writes an internal report to help inform and improve our engagement efforts.
Results
Horizon Scanning Operations and Patient Stakeholder Recruitment and Participation Data
From 7 December 2018 through 13 March 2020, analysts reviewed about 3,500 information leads and identified about 520 potential topics across the five priority areas. Ninety-two high-level trends were also identified across all areas of health care. After applying the HCHSS inclusion criteria, nomination, and voting process, 312 topics were selected and monitored; some were archived for various reasons (e.g., the intervention failed to meet clinical trial end points). Of the 312 topics, 121 topic profiles were developed, of which 105 were sent for stakeholder comment and 16 were archived. Using the HCHSS inclusion criteria and nomination process, thirty-two trends were selected, actively monitored, and sent for stakeholder comment. No trends were archived as of March 2020.
From 1 March 2019 through 13 March 2020, the PES contacted 122 organizations pertaining to the five HCHSS priority areas to request support identifying patient stakeholders. Outreach efforts led to fourteen conference calls with national advocacy organizations, international patient discussion platforms, and individual patient stakeholders. To date, fifty-four patient stakeholders have been invited to participate in the HCHSS. As of 13 March 2020, thirty-nine patients and caregivers had completed eighty-three reviews. We were unable to identify patient stakeholders with experience related to the remaining twenty-two topic profiles. Participants in our database who have yet to complete a review will be contacted when a report in their area of experience is available. Table 2 shows the number of completed patient stakeholder reviews by the clinical area.
Table 2. Patient and caregiver stakeholder reviews by disease or condition
Note: From the first review received 18 April 2019 through 13 March 2020.
This table shows the topics representing the five priority areas reviewed by patient stakeholders. The first and third columns show the specific diseases or conditions. The second and final columns show the number of reviews for each clinical disease or condition. As of March 2020, patient stakeholders had completed eighty-three reviews in the PCORI HCHSS.
Impact of Patient Stakeholder Comments on the HPDR
Patient and caregiver stakeholder perspectives inform analysts’ inclusion recommendations for the HPDR. Patient and caregiver comments either confirm other stakeholder perspectives or provide new perspectives, as seen in three topics in the May 2019 HPDR and eleven in the November 2019 HPDR (Reference Hulshizer, Carlson, Cuevas, Hudson, Lynch and Mehta9;Reference Hulshizer, Carlson, Cuevas, Druga, Hudson and Lynch10).
For example, one topic profile in the May 2019 HPDR was about a drug intervention, called idebenone, to treat Duchenne muscular dystrophy (Reference Hulshizer, Carlson, Cuevas, Hudson, Lynch and Mehta9). Six stakeholders provided comments, including a caregiver. The analyst covering this therapeutic area stated that the caregiver's detailed comments about ways this drug's potential shortcomings could affect patient and physician acceptance were enlightening. The caregiver noted that the small number of patients in trials might have prevented discovering all possible side effects of the therapy. This stakeholder also discussed the quality of life with existing treatments, which was a useful background for the analyst. Overall, the analyst reported that this perspective highly influenced his inclusion recommendation for the HPDR. Table 1 shows this result and additional examples of impact of patient stakeholder input on the HPDR.
Patient Stakeholder Experience Survey
Of the fifty-four invited patient stakeholders, thirty-nine commented on at least one report and had been surveyed as of March 2020. Of those surveyed about their experience, 92 percent (thirty-six patient stakeholders) responded. Two submissions were excluded from the interim results because one stakeholder submitted a blank survey and another indicated never receiving a report to review. Of the remaining thirty-four, respondents represented various groups: thirteen were patients, four were caregivers, three were patient advocates, and fourteen held dual roles, such as a patient and caregiver or a patient and advocate.
The most frequent responses were “agree” or “somewhat agree” (Table 3). Responses to the following four questions are of particular interest because they will help us evaluate how we prepare patient stakeholders for the review process: question one (felt prepared to review the report and complete the comment form), two (found the report easy to understand), five (was confident about the ability to provide useful comments), and seven (would recommend other patient stakeholders to participate). Also, responses helped identify the need to modify report content for this stakeholder group, such as increasing plain language use and using technical terms only when necessary (11;Reference Schriver12).
Table 3. Results of patient stakeholder experience survey
Note: This table shows the results from the patient stakeholder experience survey, which is a seven-question Likert-type survey. The seven column headings from left to right are: survey question, number of responses, percentage agree, percentage somewhat agree, percentage somewhat disagree, percentage disagree, and mean response.
Thirty-nine surveys were sent and thirty-six responses were received from 4 June 2019 through 13 March 2020. One recipient submitted a blank survey and one indicated never receiving a report to review. These responses have been excluded from the results reported here.
n, number of survey responses; N, number of respondents surveyed; PCORI HCHSS, Patient-Centered Outcomes Research Institute Health Care Horizon Scanning System.
a Response option ratings: agree = 4, somewhat agree = 3, somewhat disagree = 2, disagree = 1.
A total of 85 percent felt prepared to review the report and complete the comment form (question one), and 65 percent agreed and 26 percent somewhat agreed that the report was easy to understand (question two). A total of 68 percent agreed and 26 percent somewhat agreed that they were confident in their ability to provide useful comments (question five). Almost all (i.e., 94 percent) would recommend other patient stakeholders to participate as reviewers (question seven).
Of the respondents, 3–9 percent somewhat disagreed with questions two, three, four, and five because they found the report or comment form difficult to understand, the comment form difficult to use, or lacked confidence in their ability to provide useful comments. In responses to questions seven (recommend others to participate) and eight (any additional comments), the participants expressed difficulty in understanding the terminology used in the report and the comment form and desired more preparation before completing a review.
A total of 71 percent of respondents provided an explanation for why they would or would not recommend that other patient stakeholders participate as commenters in the HCHSS. One patient said, “This is a great opportunity for patients to advocate for themselves, as well as others. I would recommend this to anyone who has been touched by a disease.” Themes that emerged from comments recommending participation were as follows:
• A minimal time commitment (8 percent)
• A user-friendly system and process (13 percent)
• An opportunity to advocate and offer immediate feedback, be involved in research, and learn about new treatments (54 percent)
• An important way to capture patient stakeholder perspectives (29 percent)
Reasons respondents would not recommend others to participate included the following:
• Difficulty in understanding the terminology used in the report or the comment form (13 percent)
• Discomfort when reviewing the report and completing the comment form (4 percent)
• Need for more preparation or training before participation (8 percent)
• Need to identify the “right” patient stakeholders to participate, such as those with experience providing reviews (13 percent)
Lastly, we asked respondents for any additional comments about their experience as reviewers. One patient expressed that “this process seeks information in a way that frames the patient in the broader system and assesses the patient and provider in their lived environment.” Of the 44 percent who responded, comments included the following:
• A rewarding process for patients and caregivers (13 percent)
• An appreciation of the opportunity to participate as a commenter (60 percent)
• A request for more training about the patient role in the HCHSS (7 percent)
• Unfamiliarity with some of the terminologies (13 percent)
• A desire for education on interpreting statistics in drug studies (7 percent)
• A request for more details or background on the disease of interest (13 percent)
Discussion
Patient and public involvement describes a broad range of efforts to include these stakeholders in processes that guide health system decision-making (Reference Abelson, Wagner, DeJean, Boesveld, Gauvin and Bean13). HTA organizations have used various frameworks to incorporate patient and public perspectives that enhance the transparency of decision-making processes (Reference Scott and Wale14). The approach often depends on the HTA organization or project type. For example, the U.K. National Institute for Health and Care Excellence (NICE) promotes fairness and patient inclusion in decisions related to their health and well-being (Reference Pomey, Brouillard, Ganache, Lambert, Boothroyd and Collette15). The Canadian Agency for Drugs and Technologies in Health (CADTH) involves patients on working groups and committees at several levels in the assessment process for medications and medical devices (Reference Pomey, Brouillard, Ganache, Lambert, Boothroyd and Collette15). In January 2020, the U.K. National Institute for Health Research launched the Centre for Engagement and Dissemination to make research representative, relevant, and ready to use by stakeholders (16). The U.S.-based Institute for Clinical and Economic Review (ICER), focusing on cost-effectiveness studies for healthcare interventions, mainly pharmaceuticals, updated its Value Assessment Framework in January 2020 detailing procedures for including patient experiences and values in ICER evidence reviews (17).
Although the goal of patient engagement is similar for horizon scanning and HTA, the timing of engagement varies. Horizon scanning is the first step in the traditional HTA timeline and a point at which clinical data, used to consider the potential value of interventions, are typically scant. Our report is one of the first to describe and evaluate a process for including patient stakeholder perspectives at this early stage. Our results illustrate how patient perspectives influenced decisions to determine the healthcare disruption potential of topics considered for the PCORI HCHSS HPDR and what patient stakeholders thought of the review process.
In contrast, the CADTH Common Drug Review program seeks patient input to frame assessments used to develop reimbursement recommendations (Reference Berglas, Jutai, MacKean and Weeks18). Similarly, the NICE invites patient stakeholders to submit a written statement to be considered along with clinical and economic data when drafting guidance documents (Reference Staley and Doherty19).
Stakeholder perspectives have become a cornerstone of the HCHSS process. Even before this work, ECRI created and maintained relationships with stakeholder reviewers across the healthcare provider and payer continuum and included patient representatives, caregivers, and advocates in work on patient reference guides (Reference Hulshizer, Lynch, De Lurio, Erinoff, Robertson and Tipton3;20;21). Expanding our HCHSS stakeholder database to include patients, caregivers, and advocates has enhanced analysts’ insight into real-life implications of interventions in development and issues of importance to these stakeholders (e.g., patient-oriented outcomes and costs). We, like Simpson, Cook, and Miles (Reference Simpson, Cook and Miles22), have found that in an Early Awareness and Alert system, patients add valuable and unique perspectives on the quality of life, acceptability, and ease of use regarding a technology's potential impact for end users.
An important aspect of stakeholder engagement for patients is to provide a meaningful engagement opportunity in which they believe their input is useful and has an impact on the project. One way to assess this in the HCHSS is to survey patient and caregiver stakeholders and use the responses to revise and improve all aspects of engagement. Although the current sample size is small, stakeholders’ most frequently selected responses suggest they believe that their engagement was meaningful. According to HCHSS analysts, patient stakeholder perspectives have informed HPDR topic inclusion decisions by providing inputs that might not otherwise be captured, such as their lived experience with a condition.
Our survey results also indicate several ways to improve the experience by revising our preparation activities to offer more training before the review experience and using more plain language in writing topic profiles.
We also recognize that using a four-point Likert-type survey scale sometimes requires responses that might not reflect a respondent's true opinion. A five-point scale typically provides a “neither agree nor disagree” response option, which might better align with some respondents’ level of agreement with particular questions. However, the five-point scale has drawbacks including potential overuse of the middle category if one is unsure or ambivalent about the question—the mid point can be less informative (Reference Dykema, Garbarski, Wall and Edwards23;Reference Willits, Theodori and Luloff24).
Another challenge is using clinically and scientifically accurate language that is also understandable to patients and caregivers. Plain language use is essential to engaging patient stakeholders. Studies continue to show that jargon-filled writing alienates readers, even when definitions are provided (Reference Shulman, Dixon, Bullock and Colón Amill25). Supplemental File 2 compares the language from the comment form used to gather clinical, research, and health system stakeholder comments and the form used to gather patient stakeholder comments. Both groups are highly experienced, but in different ways. As such, we sought to make the plain language questions more direct and concise while maintaining a high engagement.
Additionally, preparing patient stakeholders for their tasks, providing ongoing support and feedback when necessary, and sending updates about the end product (e.g., HPDR publication) enhance engagement for all concerned. Sometimes stakeholders ask whether their first review met expectations, and the SEC works with analysts to provide individual feedback.
After the biannual HPDR is published, the PES e-mails patient stakeholders with the link to the publication to show participants how their comments were used and provides suggestions for sharing the report on social media platforms. Abelson et al. (Reference Abelson, Wagner, DeJean, Boesveld, Gauvin and Bean13) note that an enabler to meaningful public and patient involvement includes appropriate supports for patient and public committee members. These authors (Reference Abelson, Wagner, DeJean, Boesveld, Gauvin and Bean13) also found that key organizational challenges to public and patient involvement include time, financial resources, and expertise to support high-quality involvement. Simpson, Cook, and Miles (Reference Simpson, Cook and Miles22) recommend embedding patient and public engagement into early awareness and alert systems and allocating resources for the activities.
From the beginning, the HCHSS project allocated such resources (i.e., PES and SEC roles) to recruit, prepare, and communicate with stakeholders. Recruiting enough patient stakeholders with experience in specific therapeutic areas, particularly for rare disease conditions, has been the biggest challenge. Abelson et al. (Reference Abelson, Wagner, DeJean, Boesveld, Gauvin and Bean13) faced similar challenges. HCHSS outreach efforts to more than 100 national advocacy organizations and consortiums resulted in a response rate of about 11 percent, which we would like to improve. Our current participants sometimes, but infrequently, refer other patient stakeholders. In light of these recruitment challenges, the generalizability of patient stakeholder comments is limited for a given topic profile.
We are establishing a presence on social media platforms. Our posts include text and graphics to increase visibility and attract potential stakeholders. Using the top social media platforms Twitter, Facebook, and LinkedIn, we are offering content of interest to our target population, such as our February 2020 kickoff topic, which focused on Rare Disease Day. The posts are meant to be informational, serve as a recruitment tool, and provide a link to the HPDR. The recruitment content links to a brief survey to determine whether a stakeholder is interested in participating as a reviewer. We also plan to “scrape” select social media platforms for specific topic areas (e.g., new drugs) that meet HCHSS inclusion criteria and use sentiment analysis tools to examine the content for potential trends. Anecdotally, major end users of the HCHSS are PCORI, payers, and international government agencies; social media and web analytics will advance our engagement with the patient end user, too.
Conclusions
Understanding which treatments have the highest potential for disruption relies on stakeholder input to drive topic selection for each HPDR. Patient stakeholder perspectives have enhanced the selection process; most patient stakeholders would recommend others to participate. Our experience presents an upstream approach to capture patient stakeholder voices and offers methods to assess the impact of their perspectives and protocols to evaluate the meaningfulness of this type of engagement. Future research is needed to inform best practices for patient stakeholder recruitment, to identify and assess meaningful engagement opportunities, and to improve preparation activities to reduce barriers to engagement for this stakeholder group.
Supplementary material
The supplementary material for this article can be found at https://doi.org/10.1017/S026646232000207X.
Acknowledgments
The authors gratefully acknowledge the following ECRI staff for their contributions to this project: Rosemary Walker, MLIS; Jonathan Treadwell, PhD; Katherine Donahue; Helen Dunn; and Cassia Hulshizer. William Lawrence, MD, MS, and Gowri Raman, MBBS, MS, representatives from PCORI, served as contracting officers and technical representatives and provided inputs during the implementation of the HCHSS. PCORI stipulates the broad clinical areas that the HCHSS covers. PCORI does not directly participate in the day-to-day horizon scanning process, assess leads, or decide which topics to monitor or include in the HPDR. PCORI does not provide opinions regarding the potential impact of interventions.
Funding
The PCORI HCHSS is operated by ECRI under contract to PCORI. All authors were funded by PCORI (Contract No. MSA-HORIZONSCAN-ECRI-ENG-2018.7.12).
Disclaimer
All statements in this article, including its findings and conclusions, are solely those of the authors and do not necessarily represent the views of PCORI, its Board of Governors, or its Methodology Committee.
Disclosures
None of the individuals has any affiliations or financial involvement that conflict with the material presented in this manuscript.
Open access
