On March 11, 2020, the World Health Organization (WHO) declared coronavirus disease 2019 (COVID-19) a pandemic. Reference Cucinotta and Vanelli1 Efforts to control community spread of the highly transmissible virus have been variable. Reference Cho2–Reference Yan, Zhang, Wu, Zhu and Chen4 However, 1 issue remains clear: The success of any community-level mitigation strategy to control COVID-19 spread depends upon vast numbers of individuals drastically altering their day-to-day personal behavior. Where adherence to guidelines and policies such as mask wearing and physical distancing have been low, rapid community transmission and rates of illness have consistently been the highest. Reference Lewnard and Lo5
One challenge in understanding and assessing individual adherence to mitigation strategies has been the evolving guidelines on these behaviors. Initially, public health experts in the United States did not recommend the regular use of nonmedical masks in public. However, by June 2020 guidelines changed 6 as evidence had converged on several key strategies still overwhelmingly endorsed as critical: frequent hand washing; wearing masks in public; practicing good cough/sneeze hygiene; avoiding close contact with others outside the home of unknown vaccine status; and cleaning and sanitizing high-touch surfaces. 7 Additionally, COVID-19 is primarily spread by means of respiratory droplets, but the CDC and WHO still recommend frequent sanitation of surfaces, leading to some controversy. Reference Ives and Mandavilli8 These personal mitigation behaviors have been shown to reduce rates of COVID-19 community transmission, Reference Cheng, Wong, Chuang, So, Chen and Sridhar9,Reference Clase, Fu, Joseph, Beale, Dolovich and Jardine10 and yet adherence has been inconsistent. Despite vaccine availability, uptake has been low, new variants continue to arise, and breakthrough infections are common, making other mitigation factors critically important. Thus, it is crucial for researchers and public health professionals to understand individual engagement in COVID-19 mitigation behaviors, both for reducing the human and societal costs of the current pandemic, and for preparing for future pandemics. Reference Arden and Chilcot11 Such understanding begins with establishing the ability to measure the construct of individual mitigation behaviors reliably and validly.
The purpose of this research was to begin to establish a valid and reliable scale of viral respiratory illness (VRI) mitigation behaviors, particularly for the vulnerable pregnant population, with a next step to include validating this scale in a non-pregnant population. A Pandemic-Related Pregnancy Stress scale has been developed Reference Preis, Mahaffey and Lobel12 as well as a Postpartum Specific Anxiety Scale for use during global crises, Reference Silverio, Davies, Christiansen, Aparicio-García, Bramante and Chen13 but these do not directly measure COVID-19 or other VRI mitigation behaviors. As of the writing of this paper, we are aware of only 1 study that has examined the psychometric properties of evidence-based COVID-19 mitigation scales outside of measures of reliability. Reference Breakwell, Fino and Jaspal14 This scale—the COVID-19 Preventative Behaviors Index, or CPBI—was developed and evaluated in the United Kingdom, and measures perceived likelihood of following mitigation behaviors in the future as opposed to current practices and also includes items that may not be evidence-based (ie, intending to change their lifestyle, or intention to watch the news Reference Breakwell, Fino and Jaspal14 ). Other studies of COVID-19 mitigation have generally measured 1 specific behavior, like social distancing, Reference Al-Hasan, Yim and Khuntia15,Reference Reinders Folmer, Brownlee, Fine, Kuiper, Olthuis and Kooistra16 shelter-in-place orders, Reference DeFranza, Lindow, Harrison, Mishra and Mishra17 and mask-wearing. Reference Goldberg, Gustafson, Maibach, Ballew, Bergquist and Kotcher18 Some studies have utilized checklists to examine aggregate “compliance” with several different public health measures. Reference Nofal, Cacciotti and Lee19,Reference Van Rooij, de Bruijn, Reinders Folmer, Kooistra, Kuiper and Brownlee20 However, these checklists also contain behaviors that are not evidence-based (eg, participating in virtual events) and may even be actively discouraged by public health authorities (eg, stockpiling medical masks Reference Nofal, Cacciotti and Lee19 ). Incorporating as many of the evidence-based strategies as possible—while eschewing those strategies that are not—is an important facet of construct validity Reference Smith21 that has yet to be robustly investigated.
In developing a mitigation scale for VRIs like COVID-19, we evaluated the need in light of several factors, including: (1) low vaccination rates worldwide and the rise of more infectious variants indicating the COVID-19 pandemic is far from over; (2) the increasing likelihood that another VRI pandemic will emerge within the next 2 decades; and (3) the morbidity and mortality associated with more common respiratory viruses, such as seasonal influenza and respiratory syncytial virus (RSV), can be reduced through the same mitigation behaviors as those examined in the current study. We also considered nomological validity, Reference Cronbach and Meehl22 or the importance of validating the extent to which a given measure matches up to the theory or evidence base that exists, and the extent that the new scale correlates to distinct, but theoretically related, constructs of interest within a nomological net. Reference Smith21,Reference Cronbach and Meehl22 If a new measure can be validated as such, it becomes a more useful tool in evaluating public health interventions that seek to change health behaviors and the course of community spread for COVID-19 or other VRIs. Reference Smith21,Reference Hagger, Gucciardi and Chatzisarantis23
One way to examine whether a new measure is conceptually robust is to explicitly test a priori hypotheses about how an individual’s score on such a scale would correlate with other constructs within this nomological net. Reference Ziegler, Booth and Bensch24 Engaging in evidence-based mitigation behaviors is likely related to knowing and understanding the health guidelines Reference de Melo Ghisi, Abdallah, Grace, Thomas and Oh25 and the credibility of the source of those guidelines. Reference Hocevar, Metzger and Flanagin26 Likewise, we would expect that a scale measuring VRI mitigation behaviors using the COVID-19 pandemic as its test case would positively correlate with intention to receive a COVID-19 vaccination, Reference Shrank, Patrick and Brookhart27,Reference Achtymichuk, Johnson, Al Sayah and Eurich28 while expressing “adherence fatigue” for continued following of public health guidelines would likely negatively correlate to this scale. Reference Rahmandad, Lim and Sterman29 Last, studies have shown that health behaviors tend to cluster within individuals and those who engage in 1 health-promoting behavior are more likely to engage in others. Reference Poortinga30,Reference Pronk, Anderson, Crain, Martinson, O’Connor and Sherwood31 Thus, we might expect that more attention to viral respiratory illness mitigation is positively correlated with endorsing other healthful activities, like healthy nutrition, abstaining from harmful substances during pregnancy, and receiving a seasonal flu vaccine.
Finally, as of this writing, we are unaware of COVID-19 or other VRI mitigation scales that have been rigorously tested with pregnant populations specifically. Existing infectious disease theory suggests, and emerging evidence has shown, that COVID-19 infection significantly increases the risk of severe maternal, fetal, and neonatal morbidity and mortality for pregnant and postpartum populations. Reference Villar, Ariff, Gunier, Thiruvengadam, Rauch, Kholin, Roggero, Prefumo, Do Vale, Cardona-Perez and Maiz32 Likewise, data have shown that pregnant people are at high risk from complications and death due to other VRIs, like the seasonal flu. Reference Somerville, Basile, Dwyer and Kok33 This vulnerable population deserves increased research attention, including rapidly developed and validating a mitigation scale for use in ongoing studies. Furthermore, pregnancy is considered a “teachable moment” when health education is more likely to “stick” and health behaviors are more likely to be initiated. Reference Olander, Darwin, Atkinson, Smith and Gardner34 Studying COVID-19 mitigation behaviors in this population will contribute to the knowledge base for the general population, as well as provide a foundation for better understanding how pregnant people manage preventable risk from communicable diseases in general.
This study seeks to remedy the gaps in our current understanding by first developing, and then validating, a short VRI mitigation scale with a pregnant sample in the United States using the COVID-19 pandemic as its test case over 2 points in time. We then tested a wide array of behavioral and attitudinal measures that comprise a nomological net to develop a fuller picture of the construct validity for this new measure.
Methods
Participants
Pregnant Californians were offered a Web-based survey in both English and Spanish (the 2 most common languages in the state) during 2 waves of cross-sectional data collection; Wave 1 was collected between June 6 to July 29 of 2020, while Wave 2 was collected from December 24, 2020 to January 27, 2021. Surveys took approximately 20-30 min to complete, and participants were offered $10 gift cards for their participation. Participants were recruited primarily using StudyPages, a service that leverages social media to solicit participation. Currently, pregnant individuals who resided in California and were between the ages of 18 and 45 were eligible. Informed consent was obtained by means of Web-based survey. University of California, Davis’ Institutional Review Board approved the study. In total, 588 participants completed the survey during Wave 1 and 454 completed the survey during Wave 2.
Missing Data
Because this was a Web-based survey, we used several strategies to ensure high data quality. Participants who attempted to take the survey multiple times, who completed the survey in less than 10 min, or who did not reach the end of the survey instrument had their data removed before analysis. Participant age was asked twice, once at the beginning of the survey and once at the end; any participant without matching responses for these 2 items also was removed before analyses. In all, 216 participants were not included in the analytic sample as a result of these quality checks. The data also were screened for impossible values and outliers. Sporadic missing data were limited to under 10% for each variable and handled using list-wise deletion. The resulting analytic samples are comprised of 433 participants for Wave 1 and 387 participants for Wave 2. Demographic features of the analytic samples can be found in Table 1.
Table 1. Demographic characteristics across data collection wave
Note: Every participant did not respond to every item; therefore Ns do not necessarily sum to total analytic sample size
Measures
Survey items consisted of several categories: demographic features of the sample, 7 mitigation behaviors that make up the scale itself, and 13 items within the nomological net of the scale (4 hypothesized predictors of the mitigation scale, 3 items indexing attitudes or behaviors regarding the COVID-19 pandemic, and 6 health behavior items unrelated to COVID-19). See Appendix 1 for details on the specific items within these categories, and Table 2 for scale item wording.
Table 2. Summary statistics of specific viral respiratory illness mitigation strategies across waves
Data Analysis
Testing Assumptions
We examined whether each sample (wave 1 and 2) was adequate to conduct exploratory and confirmatory factor analysis. Sample sizes must be over 30035 and there must be more than 20:1 sample to variable ratio. Reference Osborne, Costello and Kellow36 Each individual item in the scale should correlate at levels of .30 or higher with at least 1 other item in the scale, average inter-item correlations should be at least r = 0.15, Bartlett’s test of sphericity should be significant (P < 0.05), and the items should have a Kaiser-Meyer-Olkin (KMO) value of .60 or higher. Reference Clark and Watson37–Reference Tabachnick and Fidell39
Exploratory Factor Analysis
We used exploratory factor analysis (EFA) using Principal Factors Analysis in Stata, Reference Carpenter40 examining solutions with 1, 2, and 3 factors across the 7-item overall scale. There were several criteria we used to choose the model that fit the data best: (a) eigenvalues for included factors should be at least 1, Reference Kaiser41 but ideally much larger Reference Russell42 ; (b) only retaining items that loaded onto a stable factor with a factor loading of more than 0.440; and (c) only retaining items that showed a communality of at least 0.3. Reference Costello and Osborne43
Confirmatory Factor Analysis
After conducting an exploratory factor analysis, we validated the chosen factor structure in the second wave of data using Confirmatory Factor Analysis (CFA). The sample size, sphericity, and KMO were tested to ensure that the sampling and data quality were adequate for these purposes. Reference Worthington and Whittaker35,Reference Osborne, Costello and Kellow36,Reference Pett, Lackey and Sullivan38–Reference Carpenter40 That model was then tested for goodness-of-fit using 3 measures: (a) comparative fit index (CFI) and Tucker-Lewis index (TLI) over 0.95 indicating “good” model fit, and CFI and TLI between 0.90 and 0.95 indicating “adequate” model fit Reference Hu and Bentler44 ; and (b) root mean square error of approximation (RMSEA) under 0.05 considered a “good” fit, and RMSEA values between 0.05 and 0.10 considered “acceptable”. Reference Browne and Cudeck45 Error terms among our items were allowed to correlate if there were theoretically sound reasons for the addition of these parameters. Finally, we report the internal consistency (using Cronbach’s α; Reference Cronbach46,Reference Bland and Altman47 ) at each wave and over both waves. Cronbach’s α > 0.8 (but <0.9) are considered “good” and Cronbach’s α between 0.6 and 0.8 are considered “acceptable” for this short, nonredundant scale. Reference Nunnally and Bernstein48,Reference Streiner49
Exploring Demographic Corollaries of the Scale
After confirming an adequate goodness-of-fit and adequate internal consistency of the scale, we described how the mitigation scores varied across demographic characteristics by using either univariate ordinary least squares regression or 1-way analysis of variance. We also described whether and how scores changed over the 2 waves of data collection.
Testing Nomological Validity
We began testing the nomological validity of the scale by examining the relationships between the scale scores and distinct constructs of interest. These constructs fall into 3 categories: (a) predictors of COVID-19 mitigation, (b) other attitudes and behaviors that relate to the COVID-19 pandemic, and (c) other (nonpandemic) related health-promoting behaviors. A significant relationship between 2 constructs was considered present if the correlation reached a P < 0.05 significance level. We also assessed the strength of the correlations, which were considered “weak” if the absolute value was less than 0.20; “moderate” if the value was between 0.20 and 0.40; and “strong” if the correlation was more than 0.40.
Results
Testing Assumptions
Both wave 1 and wave 2 samples were adequate for conducting EFA and CFA analyses. The Bartlett test of sphericity was significant for both samples: χ2 = 680.92, P < 0.001 for wave 1; χ2 = 488.79, P < 0.001 for wave 2. For wave 1, KMO = 0.71; and for wave 2, KMO = 0.70. Sample sizes of 433 and 387 provided large enough samples for factor analysis with a ratio of over 50 participants per variable in each wave. For wave 1, inter-item correlations range from 0.11 to 0.56, with all correlations significant at P < 0.05. Inter-item correlations average r = 0.29 with all other items in the scale, and each item correlated with at least 1 other item on the scale at or above r = 0.45 See Table 2 for correlations between individual items in the scale.
Exploratory Factor Analysis
Table 2 shows response rates for each item across both waves. Based on our a priori criteria, we extracted a single factor and retained all of the items. The eignenvalue for the 1-factor solution was 2.1 (2-factor solution = 0.7; 3-factor solution = 0.2). Table 3 shows results from EFA and CFA. The proportion of the scale variance explained by 1 factor is 0.89. In the 1-factor solution, all factor loadings are >0.4, and communality are all >0.3, confirming that each item shares common variance with the other items. By contrast, factor loadings for the 2nd and 3rd factors in the 2- and 3-factor solutions were largely below 0.4. Therefore, we created the scale as a unidimensional measure, labeled “Viral Respiratory Illness Mitigation Scale (VRIMS)”.
Table 3. Results of Factor Analysis
Note: Single-factor scale developed during wave 1 showed an acceptable fit for the data during wave 2: CFI = 0.97, TLI = 0.94, RMSEA = 0.061
Confirmatory Factor Analysis
Using wave 2 data (n = 387), we conducted a CFA using the sem package in Stata. A 1-factor structure was confirmed to be an adequate-to-good fit for the data. The RMSEA = 0.061, the model CFI = 0.97 and model TLI = 0.94. See Table 3 for standardized factor loadings for each item. As expected, error terms between certain items were significantly related: (a) the 2 mask-wearing items, (b) the 2 hand-washing items, and (c) the 2 surface-sanitizing items (all error term correlations significant P < 0.001). These error terms were allowed to correlate in the final CFA model.
Given the EFA and CFA results, scores on individual items were averaged, then the scale was standardized and centered for further analyses for the entire sample (n = 820). Figure 1 shows the distribution of this standardized/centered scale over the 2 waves of data collection. Scale averages shifted slightly over time: (a) wave 1 mean = −0.38, SD = 0.68; and (b) wave 2 mean = 0.39; SD = 0.53. One-way ANOVA showed that the wave 2 scale average is marginally higher than wave 1: F(817) = 3.28; P = 0.071. Over both samples, the measure showed acceptable reliability, with Cronbach’s α = 0.72 (wave 1 α = 0.74 and wave 2 α = 0.70). The measure is negatively skewed and tail-heavy at both waves.
Figure 1. VRIMS score distribution by data collection wave.
Exploring Demographic Corollaries of the Scale
VRIMS scores varied significantly by several demographic categories (see Figure 2). Older participants scored higher than younger participants (β = 0.01; P = 0.017), and Hispanic participants scored higher than non-Hispanic participants (F(813) = 18.14; P < 0.001). Participants living in more urban contexts scored marginally higher than those living in more rural contexts (β = 0.04; P = 0.066), while Black and Asian/Pacific Islander participants scored marginally higher than white and multiracial participants (F(692) = 2.34; P = 0.05). Primiparous participants scored lower than multiparous participants (F(816) = 5.83; P = 0.016). Essential worker status was not significantly related to VRIMS score.
Figure 2. Average VRIMS score by participant demographic category.
Testing Nomological Validity
See Table 4 for hypothesized and observed nomological validity correlations. Overall, the hypothesized relationships were observed in expected directions with at least P < 0.05 significance. Receiving or intending to receive both the seasonal flu vaccination and the COVID-19 vaccination were important exceptions to this finding: 29.9% of wave 2 respondents indicated that they were unsure whether they would receive the COVID-19 vaccine if it were made available to them, and although the correlation between this intention and the VRIMS was in the expected direction, it did not reach statistical significance (0.08; P = 0.19). Additionally, drinking sugary beverages was not related to the VRIMS score (r = 0.06; P = 0.11), although the other health-promoting behaviors (that do not index viral respiratory illness mitigation) were significantly related in the expect direction.
Table 4. Hypothesized and observed relationships between VRIM Scale and other indicators of nomonological validity (n = 820)
Note: Hypothesized relationship between indications and VRIMS score denoted as follows: + weak positive correlation; ++ moderate or strong positive correlation; - weak negative correlation; - - moderate or strong negative correlation; ns non-significant correlation
a Items available only for wave 2 data collection
Discussion
We developed and validated the VRIMS, a 7-item, unidimensional measure of adherence to recommended VRI mitigation behaviors, including mask wearing, social distancing, and handwashing using data from pregnant people in California across 2 waves of the COVID-19 pandemic. To our knowledge, this is the first such scale, available in English and Spanish, that measures the frequency of actual mitigation behaviors in the pregnant population or otherwise. Reference Breakwell, Fino and Jaspal14 Results demonstrated good psychometric properties, including good internal consistency and reliability across multiple waves. The scale correlated as expected with other behaviors and attitudes regarding COVID-19 and with most other health-promoting behaviors (eg, healthy diet and physical activity). Additionally, total scores were higher during the second, “surge” wave in California when the state was under a universal “stay-at-home order”, suggesting that the scale was sensitive to changes in behaviors based on shifts in rates of infection over time.
One interesting finding is that the VRIMS scores did not correlate with COVID-19 vaccination intentions. There may be a number of reasons for this, including that at the time of the survey, the COVID-19 vaccine trials had not included pregnant people, and the emergency use authorization (EUA) did not extend to this group. Reference Heath, Le Doare and Khalil50 At the time of the writing of this study, however, over 100,000 pregnant people had received the vaccine 51 after an initial study showed its safety in 10,000 pregnant people. Reference Feuer52 This policy shift may change the association of COVID-19 vaccination intentions and other mitigation strategies as captured in our scale, and future studies should investigate this. Conversely, other studies have shown that there is general vaccine hesitancy among pregnant people due to debunked research that has linked prenatal and early infancy vaccination to childhood disability and death, Reference Rosso, Massimi, Pitini, Nardi, Baccolini and Marzuillo53 which may be exacerbated by the unique circumstances under which the COVID-19 vaccine was developed. Public unease over the “fast pace” of production, distrust of pharmaceutical companies, and associated safety concerns may further confound the relationships between vaccination and other mitigation strategies. Reference Rosenbaum54 Given the important role of vaccination in addressing not only this pandemic, but future respiratory pandemics and more common VRIs such as flu, future research should better elucidate vaccination decisions in the context of other modifiable risk-reducing behaviors.
While our study found that the VRIMS is a valid and reliable tool, future researchers should extend use of the scale and establish its validity in other populations and settings. We assessed mitigation behaviors at the individual level, independent of specific local management of the pandemic, including such factors as local case rates, industry closures, contact tracing, mass testing, and financial means to adhere to mitigation strategies. Reference Anderson, Heesterbeek, Klinkenberg and Hollingsworth55,Reference Ebrahim, Ahmed, Gozzer, Schlagenhauf and Memish56 Scale properties could potentially be different in localities where public health measures and individual supports for adherence were greater than in the United States. Furthermore, our data were exclusively from pregnant people in California. Other studies should investigate the scale’s properties across different states, countries, and cultures, among non-pregnant people, and in nationally representative samples. Additionally, because this survey was conducted online, we may not have reached participants without stable Internet connectivity; in-person and phone-based surveys to validate other modalities may therefore be a fruitful next step.
Future studies should also test discriminant and predictive validity. For example, studies could examine how the CPBI, which measures likelihood of adhering to mitigation behaviors, compares with the VRIMS. Reference Breakwell, Fino and Jaspal14 Likewise, our scale could be used to assess whether interventions to improve COVID-19 or other VRI mitigation behaviors result in expected changes in item and total scores. Studies also could test the sensitivity and specificity of the scale’s total score for predicting exposure risk to better target public health messaging. While VRIMS is available in both English and Spanish, we did not have a large enough sample to specifically validate a Spanish version, and this should be pursued. Last, future studies should establish face validity by asking health-care and public health professionals how well the items represent the construct of mitigation as operationalized. While we have relied on evidence-based public health agency guidelines in the creation of the scale items to address this concern, there may be other, missing facets of VRI mitigation behaviors that could enhance utility of the scale for specific provider and public health audiences.
Our scale has the potential to benefit research and clinical practice now and in the future. The VRIMS is short, nonredundant, evidence-based, and unidimensional with good psychometrics. This makes future inclusion of the VRIMS in studies aimed at understanding and improving mitigation behaviors for COVID-19 and other VRIs simple without increasing participant burden. Likewise, as we better establish the utility of the VRIMS for identifying risk categories, clinicians may use the scale for targeted patient education. Given the rise of COVID-19 variants and lower than desired vaccination rates for other VRIs (eg, measles, seasonal flu), our scale could prove useful in future public health crises both locally and globally because the evidence-based guidelines for mitigating transmission of these illnesses are the same. Reference Swerdlow and Finelli57
Conclusions
The VRIMS is a short, unidimensional measure of evidence-based individual mitigation behaviors. The scale shows acceptable internal reliability, adequate psychometric properties, demonstrable construct and nomological validity, and consistency over time for a population of pregnant Californians. It is also easy to interpret and use, and could be used immediately as a monitoring or evaluation tool to aid in policy and intervention development, having shown good psychometric qualities across both a “surge” and “nonsurge” pandemic context. Future research can and should continue to test and refine this scale to use in the ongoing context of the COVID-19 pandemic, for future pandemics, and for outbreaks of common, seasonal respiratory illnesses such as influenza and RSV.
Supplementary Material
To view supplementary material for this article, 344 please visit https://doi.org/10.1017/dmp.2022.103.
Acknowledgments
The authors acknowledge Sharon Tafolla, who assisted with translation of the survey into Spanish, and Nikita Satish for assisting with literature review and editorial assistance.
Funding statement
Funding for the current study was provided by Hatch Project #CA-D-HCE-2582-H, Addressing Multifactorial Influences on Pregnancy Outcomes to Promote Health Equity. Funding for Dr. Whipps was provided by 5R01NR017659 (Simmons PI).
