Background
The lack of equitable representation in research compounds health inequities and has serious consequences [Reference Bibbins-Domingo and Helman1]. Both mistrust and structural problems in how we conduct research to contribute to lack of representation in research, underrepresented people, and communities are generally willing to participate in research if respectfully asked.
A key function of Clinical Translational Science (CTS) Centers is to build infrastructure and methods that promote equitable inclusion of socially and economically marginalized and medically underserved people and communities in translational research [Reference Yousefi Nooraie, Kwan and Cohn2]. Nationally, CTS centers are using novel community engagement methods to design more acceptable and feasible research, build relationships and establish trust, and work to ensure that research aligns with the needs of vulnerable and underrepresented people [Reference Zimmerman E.3]. CTS Centers’ success is partially judged by their ability to promote more inclusive and representative participation in research [Reference Ortiz, Nash and Shea4,5]. The novel integration of population health data can further add to CTS Centers’ ability to design and recruit for more equitable research.
In 2019, the Agency for Healthcare Research and Quality funded six grantees to provide practice facilitation to 125 primary care practices to better implement the U.S. Preventive Services Task Force (USPSTF) recommendation to screen for unhealthy alcohol use and brief counseling and treatment for patients with hazardous drinking [6]. This was done through the EvidenceNow initiative, a series of funding announcements to promote the uptake of guidelines and evidence-based practice into routine care [7].
Unhealthy alcohol use is the third leading cause of preventable death in the USA [8]. There is strong evidence to demonstrate the benefits of screening and counseling for unhealthy alcohol use in primary care [Reference OʼConnor, Perdue and Senger9], hence the USPSTF recommends that primary care clinicians provide this service routinely for all adults [Reference Curry and Krist10]. While highly effective and feasible to deliver in primary care—taking one to two sessions and 2–30 thirty minutes in implementation trials reviewed by the USPSTF—these services are poorly delivered. Most clinicians report that they do not routinely screen or provide counseling and treatment interventions, and most patients do not recall being asked about alcohol use or receive feedback on their drinking habits [Reference McKnight-Eily, Liu and Brewer11–Reference Weisner, Campbell and Altschuler14].
Despite the need and value of promoting this preventive service, enrolling 125 busy primary care practices for training to better deliver under-delivered services is an audacious task that is reliant on established relationships with practices. All grantees in this initiative were primarily CTS centers and practice-based research networks. Adding to the complexity of recruiting such a large practice sample, shortly after this initiative started the COVID pandemic hit. Primary care practices—already in survival mode—eschewed any task that was not essential to addressing COVID or their core responsibilities, like participating in research [Reference Krist, DeVoe, Cheng, Ehrlich and Jones15]; concurrently, the COVID pandemic made unhealthy alcohol use worse, further intensifying the need for these services. Prior to the pandemic 20–25% of US adults drank at unhealthy levels and 14% had alcohol use disorder, and these rates substantially increased during the pandemic [Reference Grant, Goldstein and Saha16,Reference Roberts, Rogers and Mason17].
This manuscript describes the novel use of a comprehensive primary care clinician and practice database, the Virginia All-Payers Claims Database (APCD), and an analytic database and mapping tool with community-level socio-ecological information, called HealthLandscape Virginia (HLVA), to ensure successful and inclusive recruitment of primary care practices for an implementation study to address unhealthy alcohol use [18–20].
Methods
This is a prospective observational analysis of the recruitment strategy for a practice-level randomized controlled trial to provide early versus delayed practice facilitation for improving screening for unhealthy alcohol use and brief counseling interventions for hazardous drinking. We prospectively analyzed the inclusiveness of the practices and communities recruited for participation and iteratively adapted our recruitment strategies to ensure all communities in Virginia would be represented and benefit.
Practices were approached to participate in this study and provided the details of the intervention. Specifically, practices were randomized to immediate versus 6-month delayed practice facilitation. Each practice was asked to identify one to three practice champions to participate in up to four learning collaboratives to develop a standard approach to implementing the screening and counseling recommendation. Then the champions led up to three practice-wide sessions to implement the screening and counseling workflow. Champions and practices had access to a board spectrum of patient and community-level resources [21]. We conducted a chart review and mailed a survey to 60 patients randomly selected at baseline, 3 months, and 6 months postintervention to assess the impact of the practice facilitation intervention. This manuscript focuses on the use of baseline results for recruitment.
The methods for the practice-level pragmatic trial have been reported previously [Reference Huffstetler, Kuzel and Sabo22]. This study was approved by the Virginia Commonwealth Institutional Review Board (HM20016728).
Recruitment Approach. The EvidenceNow Request for Applications for this funding initiative (RFA-HS-18-002) defined recruitment of 125 practices and providing practice facilitation for screening and counseling for unhealthy alcohol use as a key requirement for applicants and the primary measure of success [23]. To address this requirement, in our proposal we described a practice recruitment strategy that would leverage existing relationships and connections with our primary care practice-based research network, the Virginia Ambulatory Care Outcomes Network (ACORN), and our five Virginia Commonwealth University and VirginiaTech Carilion family medicine residency training centers. Both ACORN and the residencies are distributed throughout the state and approximately 40% of graduating residents continue to practice in Virginia after matriculation. We defined five regional hubs centered around each residency to recruit 30 local practices for participation; we planned to oversample in the event practices dropped out of the study.
With the onset of the COVID pandemic, we quickly realized that the stresses experienced by our training centers and ACORN network practices would make it difficult to meet the required 125 practices for participation. We also realized that the practices initially recruited were not fully representative of primary care or the communities primary care practices serve in Virginia, so we created a process to use real-world data to improve and monitor recruitment. This process used a novel internal database of all primary care practices and clinicians in Virginia to identify the practices participating in our study and who could be recruited for participation, the APCD to identify the communities impacted by practices participating of not participating, and HLVA to visually and geographically understand the characteristics of the communities included and not included in our study.
This resulted in four unique recruitment approaches:
-
Approach 1 (Training Approach) – as described in our proposal, we created five hubs at the residency practices throughout the state. The residencies participated in the study and a faculty champion contacted graduates and local practices about study participation.
-
Approach 2 (Health System and Professional Organization Approach) – we recruited leadership from health systems in Virginia who agreed to disseminate recruitment information to their practices. The Virginia primary care specialty societies agreed to send recruitment material to their membership.
-
Approach 3 (Targeted Approach) – using the APCD and primary care database, we identified primary care practices in communities with low inclusion from prior recruitment efforts and emailed, mailed, or called the practices about study participation. These practices were not part of health systems that agreed to participate in Approach 2 and had not participated in prior ACORN studies.
-
Approach 4 (Concluding Approach) – we repeated the actions in Approaches 1–3.
Outcomes. The outcomes for this analysis include the number of practices recruited using each approach and the characteristics of the practices, communities, and patients included in the study for each recruitment approach.
Data Sources. We used four data sources to prospectively inform our recruitment process and to assess the outcomes of the four recruitment approaches. The first data source is the ACORN primary care clinician and practice registry for Virginia. This data source is supported and maintained by ACORN, the Wright Center for Clinical and Translational Research, and the Department of Medical Assistance Services to inform ACORN activities, support translational research efforts, and track the role of primary care with Medicaid Expansion [Reference Krist, Brooks and Donahue24,Reference Brooks, Huffstetler and Britz25]. The methods used to create the database have been described previously [Reference Huffstetler, Sabo and Lavallee26]. To identify the primary care workforce, we annually query the National Plan and Provider Enumeration System to identify specialty, verify the presence of an active claim in the APCD, and verify at least 10 wellness visits for nonfamily medicine clinicians in the APCD. We identify new clinicians, any change in claims activity (i.e., absence of claims or new wellness claims for clinicians), or any change in location compared to what is currently in the ACORN database. Clinician and practice data are verified manually with online information.
The second data source is Virginia’s APCD. We use this to identify the primary care clinician workforce (described above) and to identify the census tract that patients live in for the participating practices. Virginia has a robust and comprehensive APCD. It was established in 2013 and contains data on roughly 65% (5.5 million out of 8.5 total) of Virginians, including 100% of Medicare and Medicaid claims and 50% of commercial claims [27]. The third data source is the HLVA Community Vital signs library [20]. This includes census tract-level variables on a range of socioecological factors such as poverty, education, race/ethnicity, internet access, transportation, housing foreclosures and vacancies, residential segregation, small area measures of chronic conditions, access to care, inequality, and composite measures of deprivation and vulnerability (social deprivation index, social vulnerability index, and index of deep disadvantage). Data come from multiple sources such as the American Community Survey, 500 Cities Project, Behavioral Risk Factor Surveillance System, National Health and Nutrition Examination Survey, and more [28–30]. The demographic data used as the comparison for our analysis was primarily from the American Community Survey. It describes the demographics for Virginians of all ages, except marital status and education are for residents ages 15 years and older (note our study sample is patients ages18–79 years).
The final data source is the chart review and patient survey responses for baseline patients included in the practice facilitation trial. At enrollment, each practice generated a list of patients aged 18 to 79 years seen in the prior 3 months that would be eligible to receive screening and counseling for unhealthy alcohol use. We randomly selected 60 patients for inclusion and our research coordinator conducted a chart review to assess demographic characteristics (age, sex, race, ethnicity, insurance type, and preferred language). The same patients were also mailed a survey using a modified Dillman technique to assess demographic information not routinely documented in the medical record, such as marital status and education, as well as outcomes from our clinical trial.
Analysis. We compared the practice characteristics of enrolled practices to all the practices in the Virginia primary care database, and the patient characteristics from the chart review and patient survey to the overall demographics of Virginia using the 2021 American Community Survey and 2022 Medicaid and CHIP data [31,32]. Additionally, practice rurality was determined using the patient zip codes and the locale codes created by the Education Demographic and Geographic Estimates (EDGE) Program [Reference Geverdt33]. We defined each overall practice and patient characteristic as similar to the state average if the value was between 80% and 120% of the state average. Likewise, we defined the practice and patient characteristics for each of the four approach periods as similar to the overall study sample if its value was between 80% and 120% of the overall study sample average. To map the communities potentially impacted by our study, we used the APCD to identify each census tract where one or more patients lived that was seen in the recruited practice in 2019. Practice and patient characteristics were summarized with frequencies and percentages. Chi-squared tests and Fisher’s exact tests were used to determine if there were differences in characteristics across approaches.
The GIS software ArcGIS Pro 3.0.3, developed by the Environmental Systems Research Institute (ESRI), was used for creating maps. We first imported and created a geodatabase in ArcGIS that contained a specific approach (i.e., approaches 1–4 and overall patient density (per 2,500)) and the corresponding patients for each census tract within Virginia.
We then conducted a spatial join between this geodatabase and the shapefile of Virginia based on the unique census tract field (U.S. Census Bureau, 2019). Since the number of patients was heavily right skewed for all approaches, we manually defined classifications, though for easier comparison, we incorporated the same cutoff values for the maps of approaches 1 through 4 [34]. All statistical analyses were performed at a significance level of 0.05 using R version 4.1.0.
Results
Using our four recruitment approaches, we enrolled 76 practices for study participation (Table 1). The residency training approach (#1) enrolled 24 practices with patients living in 1681 census tracts; the health system approach (#2) enrolled 17 practices with patients living in 1126 census tracts; the targeted approach (#3) enrolled 22 practices with patients living in 892 census tracts; and the final push approach (#4) enrolled 13 practices living in 1201 census tracts. As a result, the final practice sample cared for patients living in 97.3% (1844 of 1907) of Virginia’s census tracts (Fig. 1).
Figure 1. Distribution of where the study practices’ patients live. these maps show the number of people that live in each census tract of virginia that had an office visit with one of the recruited study practices in each of the four recruitment approaches and for the overall study sample.
Table 1. Characteristics of practices enrolled in the study
p value compares the practice demographics from wave to wave using a two-tailed t test.
For Approach #1–4 cells, green is 20% higher than All Practice characteristic and orange is 20% lower.
For All Practices cells, green is 20% higher than All Practices in Virginia and orange is 20% lower.
a Percentages based on responses to the Virginia Primary Care Survey responses [18]. The survey was mailed to Virginia’s 2,296 primary care practices; 445 of 2,026 practices for adults (22%) completed the survey.
b Fisher Exact test was used as Chi-squared was not sufficient.
The practices enrolled for study participation were generally representative of primary care in Virginia with some important exceptions (Table 1). Compared with all of Virginia’s primary care practices, the practices participating in our study were less likely to be urban (14.5% vs 22.5%), more likely to be health system owned (71.1% vs 42.2%) or a community health center (7.9% vs 1.2%), and more likely to be academically oriented including patient-centered medical home designation (52.6% vs 63.1%) or part of an accountable care organization (63.2% vs 44.6%). Across approaches there were significant differences in practices enrolled between practice type (p = 0.029) and practice ownership (p = 0.004). The training approach recruited more suburban and clinician-owned practices than the other approaches; the health system and professional organization approach more mixed primary-specialty care, urban, rural, and community health center practices; the targeted approach more unique practices from nonrepresented health system and academically oriented practices; and the concluding approach added some obstetrics-gynecology and direct primary care practices.
The characteristics of the overall sample of study patients were very similar to the demographics of Virginia with respect to sex, ethnicity, race, commercial or Medicaid insurance status, and education (Table 2). Our sample only differed by including fewer patients who were Asian (5.7% vs 7.2%), whose preferred language was Spanish (4.2% vs 7.1%) or a language other than English (1.4% vs 8.3%), had Medicaid (13.7% vs 22.5% or Tricare (0.9% vs 1.8%) for insurance, had a high school education (19% vs 23.9%), or were never married (18.9% vs 32%) or were separated 3.7% vs 5.5%). Across recruitment waves, there were statistically significant differences in ethnicity (p < 0.001), race (p < 0.001), language (p < 0.001), insurance (p < 0.001), education (p < 0.001), and marital status (p < 0.001). The training approach recruited more Black, Asian, and educated patients; the health system and professional organization approach more patients with Medicaid insurance and never married or divorced people; the targeted approach more Hispanic, Spanish or other language speaking, self-pay and uninsured, and less than high school educated patients; and the concluding approach more patients with Medicaid insurance and other language speaking (Table 2).
Table 2. Demographic characteristics of patients included in the study a
p value compares the patient characteristics from wave to wave using a two-tailed t test.
For Approach #1–4 cells, green is 20% higher than All Practice characteristic and orange is 20% lower.
For All Practices cells, green is 20% higher than All Practices in Virginia and orange is 20% lower.
a Patient level data available for 65 of 76 participating practices.
b Values from the 2021 American Community Survey (census) of Virginia for residents of all ages, except marital status and education which is for residents age 15 years and older [31].
c Comparison of demographic characteristics of All Patients Included in Study and characteristics of Virginia Residents.
d Values derived from chart review of patient’s electronic medical record.
e Values reported by patients on survey.
f Based on 2022 Medicaid and CHIP data [32].
g Fisher exact test was used as Chi-squared was not sufficient.
Discussion
Our use of real-world data on primary care practices and the communities they care for resulted in a study sample that was highly representative of Virginia and inclusive of most communities in the Commonwealth. While the final sample of 76 practices was below the targeted sample size of 125 practices, it was substantially more practices than others were able to recruit during the COVID pandemic, an especially difficult time to engage any primary care practices in research. More importantly, we were able to demonstrate that changing the routine practice patterns of these 76 practices will have a broad impact throughout the state. Not all practices that were contacted through the four approaches agreed to participate – in fact most declined – but the process expanded our ACORN network and improved our capacity to recruit for future translational studies.
The observed differences in our patient sample versus the demographics of Virginia are largely reflective of the patients seen in primary care and the difference in our study sample (people ages 18–79 years) and the demographic data on Virginia residents which reports on residents of all ages. Older people who are married and widowed are more likely to seek medical care than younger single people. We did not recruit practices in the Veterans Administration or practices so we would expect to have lower proportions of people with Tricare insurance. While we did oversample community health centers for recruitment and purposely targeted practices that served non-English speaking patients in our Targeted Approach, we still had fewer Spanish or non-English speaking patients in our final sample. These practices may take more effort for recruitment into research studies. Our proportion of patients with Medicaid is in line with the proportion of the population with Medicaid at the start of the study, as the number of Virginians with Medicaid doubled between 2019 and 2022 because of Medicaid Expansion [Reference Norris35]. The state rate of people with Medicaid may also be higher because it includes children, who are more likely to be on Medicaid than those ages 18–79 years.
An important part of our equitable recruitment process was knowing not only about the characteristics of the practices we approached for participation but also knowing about the communities they served. In many cases, this had a greater influence on the characteristics of the patients included in the study than the practice characteristics. Knowing about the practices not included in our study sample was also essential to adapting our recruitment processes, which informed us of the people and communities our study could be expected to miss. This helped our team to prioritize new practices to recruit to ACORN, which not only benefitted this study, but also future ACORN efforts. Historically, it is difficult to identify the primary care workforce, let alone know which clinicians practice in which practices [36,Reference McLafferty, Freeman and Barrett37]. Practice-based research networks, like ACORN, have relationships with and know about member practices to create a primary care practice laboratory [Reference Westfall, Mold and Fagnan38–Reference Mold, Peterson, Mold and Peterson42], but often do not know about nonmember practices. Our use of licensure and claims data provided an understanding of these nonmember practices.
Limitations. A key limitation of the approach that we describe is that it may not be generalizable. Currently, translational scientists do not have universal access to a primary care clinician and practice database. However, the methods we used to create the database could be replicated in other states, particularly by CTS Centers and practice-based research networks that often have connections and an understanding of the communities and primary care practices in their region. Additionally, not all states maintain an APCD like Virginia [43], although other similar data sources could be leveraged in these regions. Additionally, we did not assure representativeness for all people and communities. Our dataset allowed us to address geographic, racial, ethnic, educational, insurance status, and poverty measures for tracking and promoting equitable participation. Future data efforts will better include in research other socially and economically marginalized and medically underserved people and communities.
Conclusion
Novel use of real-world data as part of a deliberative approach to equitable recruitment of practices for dissemination and implementation trials can ensure inclusion of people and communities that have not historically participated in or benefitted from research. Creating and supporting the use of these data tools should be part of the translational science infrastructure that CTS centers support.
Acknowledgments
This work was supported by a grant from the Agency for Healthcare Research and Quality (1R18HS027077, PI: Alex H. Krist), as well and the Community Engagement Core and the Biostatistics, Epidemiology and Research Design (BERD) Core of the Wright Regional Center for Clinical and Translational Science (award No. UL1TR002649 (PI: F. Gerald Moeller).
The authors thank our health system partners (VCU, Inova, Carilion, Riverside, Bon Secours Mercy, Sentara), professional society partners (Virginia Academy of Family Physicians, Medical Society of Virginia, Virginia Center for Health Innovation), local champions (Jon Winters, John Youk, Mark Ryan, Scott Strayer, Ashley Edwards, JP Verderese, and Patricia Wessler), and data partners (Wright Center for Clinical and Translational Research, Virginia Health Information) for supporting our recruitment processes. We would also like to thank Evan French for support with querying the Virginia APCD, Paulette Lail Kashiri for recruitment support, and My Nguyen for analytics support.
Disclosures
Drs. Krist and Epling served on the U.S. Preventive Services Task Force for the 2020 Recommendation for Unhealthy Alcohol Use in Adolescents and Adults: Screening and Behavioral Counseling Interventions Recommendation. The remaining authors have no conflicts of interest to declare.
Open access
