Throughout much of the COVID-19 pandemic, many nursing homes have relied on testing staff and residents for severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) to help prevent virus spread and reduce infection risk. Reference McGarry, Gandhi and Barnett1 It is particularly important in nursing homes because residents are at greater risk for infection and more severe COVID-19 outcomes, given their age, comorbidities, and congregate living setting. Reference Wasserman, Ouslander and Lam2 Because residents have limited interactions outside the nursing home, staff and visitors are the main ways SARS-CoV-2 is introduced. Often, staff are underresourced, work multiple jobs, have low levels of education, insufficient paid sick time, and are accustomed to working while ill, Reference Ayalon3–Reference White, Wetle, Reddy and Baier5 making it difficult to speak up when experiencing COVID-19 symptoms. Thus, routine testing strategies coupled with assurances of paid sick leave may provide solutions for containing SARS-CoV-2 spread.
To date, implementation of nursing-home staff testing strategies has varied, Reference McGarry, Gandhi and Barnett1,Reference Blain, Rolland and Tuaillon6,Reference Tang, Bigelow and Sheikh7 partly because the value of such strategies has not been quantified. For example, in August 2020, the Centers for Medicare & Medicaid Services (CMS) required at least once weekly routine testing of staff. This policy was revised in September 2022 given the widespread adoption of vaccines. 8 Implementing testing requires time, effort, and money, which are not trivial because nursing homes have constrained resources. Thus, determining the value of testing can help guide its implementation. Understanding how frequently nursing home staff should be tested, which test should be used [ie, antigen or polymerase chain reaction (PCR)], and how the value of testing can vary based on several factors (eg, SARS-CoV-2 activity in the community), can help determine effective testing strategies. Therefore, to inform nursing home care, we developed an agent-based model (ABM) representing a typical nursing home and SARS-CoV-2 spread to determine the clinical and economic value of various COVID-19 screening and isolation strategies under various circumstances.
Methods
Agent-based model overview
We developed an ABM in Python software representing a typical nursing home with 100 residents, its staff, their interactions with each other, SARS-CoV-2 spread, potential health and economic outcomes, Reference Bartsch, Ferguson and McKinnell9–Reference Wedlock, O’Shea and Conte15 and testing. The nursing home model consists of 50 occupied double rooms, each in a housing pod, representing a physical location of 10 rooms. We represented 3 types of nursing-home staff: resident-facing staff providing routine care (eg, certified nursing assistants, licensed vocational or registered nurses, and environmental services workers), resident-facing staff providing specialty care (eg, physical, occupational, and speech therapists, and wound care nurses), and non–resident-facing staff (eg, medical records and office and administrative support). Routine-care staff had an assigned housing pod because they are generally assigned to the same nursing home area or section and its residents for continuity of care. The model advances in discrete, 1-day time steps for a typical winter season (December–February) when respiratory viruses tend to spread. Appendix Table 1 shows model input parameters, values, and sources.
Agent movement and mixing
Figure 1a shows how residents and staff mixed and moved throughout the nursing home in the model. Each day, agents within the nursing home interacted with each other (Appendix Table 2 online). A resident’s degree of interaction varied based on his or her location (eg, roommates), social groups and connections, and assigned staff (eg, routine care, specialty care). A resident’s social groups and connections involved either mixing or not mixing with other residents (eg, resident is nonmobile or has limited interaction). Routine-care staff interacted with residents in their assigned housing pods, and specialty-care staff interacted with postacute-care residents (those with a length-of-stay <100 days) weekly. Interactions between staff members varied based on staff type. Agents mixed each day until leaving the nursing home (eg, length-of-stay elapses, hospitalized, or dies from COVID-19) or leaving their job not due to COVID-19 (94% annual turnover rate based on CMS data of 492 million nurse shifts Reference Gandhi, Yu and Grabowski16 ). Each day, new individuals entered the nursing home such that the number of new admissions equaled the number of resident deaths and bed turnovers and new staff equaled the number of staff deaths and turnovers.
Figure 1. Model overview: (A) nursing home structure and agent mixing and movement; (B) nursing home resident SARS-CoV-2 infection pathway and testing and isolation interventions; and (C) nursing home staff SARS-CoV-2 infection pathway and testing and isolation interventions.
SARS-CoV-2 transmission
Figure 1 (b and c) shows the 7 mutually exclusive SARS-CoV-2 states that each agent could be in on any given day and how agents moved through them. Hospitalized residents temporarily left the nursing home and could not transmit to others in the facility. Staff convalesced at home.
Each day, agents interacted with each other, and an infectious person could potentially transmit SARS-CoV-2 to a susceptible person. If a susceptible person came into effective contact with an infectious person (ie, interacted and SARS-CoV-2 was transmitted), he or she became exposed and was infected but was not yet contagious. The following equation governed if a susceptible resident became infected:
1 – [(1 – Daily Contact ProbabilityResident to Routine Staff
× Transmission Probability)No. of Infected Routine Staff
× (1 – Daily Contact ProbabilityResident to Specialty Staff
× Transmission Probability)No. of Infected Specialty Staff
× (1 – Daily Contact ProbabilitySocial Resident to Other Social Residents
× Transmission Probability)No. of Infected Social Residents
× (1 – Daily Contact ProbabilityResident to Roommate
× Transmission Probability)No. of Infected Roommates]
Because the transmission probability between any 2 individuals was unknown, we calibrated this parameter to achieve a reproduction number of 9 (R0; average number of secondary cases generated by a single infectious case in a fully susceptible population), corresponding to the omicron variant. Reference Liu and Rocklöv17 To do this, we simulated SARS-CoV-2 spread assuming homogeneous mixing (average daily contact probability, 3.7%). We determined that a 15% transmission probability per contact with an infectious agent resulted in an R0 of 9.
Each infected individual became infectious up to 2 days prior to symptom onset, regardless of symptom development and had a probability of being asymptomatic (nonovert symptoms) or symptomatic (overt symptoms). After agents recovered, they were unable to be reinfected for the remainder of the simulation (immunity is assumed to last for ≥90 days 18 ).
Agent COVID-19 health outcomes
Each symptomatic individual started with a mild infection and had a probability of progressing to severe disease requiring hospitalization (a distribution draw determines when he or she was hospitalized). On hospital admission, each agent drew a length of stay from a distribution and remained hospitalized for that duration. A hospitalized agent had probabilities of intensive care unit admission and COVID-19–associated mortality. If the resident survived and their length of stay was ≤10 days (ie, the bed was held for ≤10 days), he or she returned to the nursing home.
Ongoing prevention and control measures: Vaccination and face mask use
As previously described, Reference Bartsch, O’Shea and Chin10–Reference Bartsch, Wedlock and O’Shea13 vaccination decreased an individual’s risk of becoming infected during the simulation (by 1 − vaccine efficacy against infection). Once infected, a vaccinated individual had a lower probability (1 – vaccine efficacy against severe disease) of severe outcomes requiring hospitalization. Given robust vaccination campaigns in nursing homes, individuals who were vaccinated either could have received the primary series plus a booster or could have also received the bivalent booster (within the last 6 months).
Each day, staff members wore surgical masks, which decreased the probability of transmission by 1 minus the effectiveness of face masks [face mask efficacy multiplied by compliance with their use multiplied by percent of time masked (accounting for unmasked mealtimes)].
COVID-19 testing
In the model, nursing home staff are tested at set intervals as well as on demand when reporting possible COVID-19 symptoms (ie, symptom-based testing). The antigen or PCR test had an associated sensitivity, specificity, turnaround time, and cost. When undergoing testing at set intervals, staff awaiting results still worked, maintaining surgical face mask use. However, staff undergoing symptom-based testing wore N95 respirators while awaiting test results and only continued working if their test was negative. Staff who tested positive stayed home for at least 6 days or until they recovered or tested negative, following Centers for Disease Control and Prevention (CDC) guidance. 19 During this time, they were not replaced by other staff.
Residents underwent symptom-based testing when presenting with symptoms (eg, due to COVID-19, other respiratory pathogens). Residents with symptoms had a probability of informing or demonstrating to staff that they had symptoms. Residents undergoing symptom-based testing were quarantined while awaiting test results. Residents testing positive were isolated for 10 days. The roommates of residents testing positive were quarantined for 10 days due to potential exposure. Resident isolation and quarantine required N95 respirator use by staff. Additionally, according to national guidance, all agents who test positive were not eligible for repeated PCR testing within 90 days because individuals may remain positive on molecular tests but not be infectious. 8 Furthermore, false-positive PCR results are uncommon.
Costs and economic outcomes
Each person accrued relevant direct medical costs, productivity losses, and health effects as he or she traveled through the model. These costs then contributed to the calculation of cost-effectiveness from the CMS (a type of third-party payer), total third-party payer, and societal perspectives (described in the Appendix online). For each scenario, we calculated the incremental cost-effectiveness ratio (ICER) as follows:
$$\rm ICER= (Cost_{Testing}-Cost_{NoTesting}) \div (Health\ Effects_{Testing}-Health\ Effects_{No Testing})$$
where health effects were measured in quality-adjusted life-years (QALYs) lost due to COVID-19. Each COVID-19 case lost QALYs based on their age-dependent healthy QALY value and severity-specific utility weights for their infection duration. Death resulted in the loss of the net present value of QALYs for the remainder of an individual’s lifetime. 20 We considered testing to be cost-effective if the ICER was ≤$50,000 per QALY. All costs are reported in 2023 values.
Experimental scenarios
Experiments consisted of 100 trials, we compared testing versus no testing in the nursing home, assuming the omicron variant and winter 2023–2024 vaccination and face mask use conditions. Scenarios consisted of varying test and SARS-CoV-2 parameters including the test type (antigen, PCR) with its associated performance characteristics (sensitivity and specificity, cost, turnaround time), testing frequency (twice per week to once per week to every other week in addition to testing when residents and staff reported or had discoverable symptoms), and the risk of COVID-19 in the community (10%–50% over 3 months). We also varied the probability of having more severe outcomes (eg, hospitalization or death) to account for new variants. Sensitivity analyses varied face mask compliance (50%–90%), bivalent booster coverage among residents (30%–70%) and staff (10%–50%), and vaccine efficacy against infection (primary series and booster, 10%–60%; bivalent booster, 25%–75%). In additional scenarios, we assumed that staff did not routinely use face masks and only used them when interacting with quarantined and/or isolated residents.
Results
Winter 2023–2024 omicron variant situation
Our first set of scenarios represented the winter 2023–2024 situation and current testing strategy of symptom-based antigen testing when staff and residents reported or had discoverable symptoms. Appendix Table 1 shows the parameter values for these scenarios which reflect highly vaccinated staff and resident populations and high compliance with face masks. We set the risk of COVID-19 in the community at 30% over the 3-month winter season (0.4% per day) with 10% of the population with prior immunity.
In this context, we first compared symptom-based antigen testing to no testing, assuming that detecting infectious persons accelerated isolation at home (staff) or in their room (residents) and therefore reduced spread. Symptom-based testing averted only a median of 1 COVID-19 case (range, −23 to 28) among staff and 3.5 cases (range, −20 to 34) among residents compared to no testing when only 50% of those experiencing symptoms were reported or discovered and were subsequently tested, leaving half to continue spreading to others.
The low number of averted cases from symptom-based testing generated minimal averted hospitalizations and deaths given the mild severity of the omicron variant: 0 (range, −5, 7) averted hospitalizations, 0 (range, −2 to 2) averted deaths, and 0.004 (range, −38.39 to 36.65) averted lost QALYs. Overall, over the 3-month winter season, the averted clinical outcomes saved $191 (range, −$99,923 to $204,903) in direct medical costs due to illness for all residents and staff.
However, testing costs money, and the costs of implementing testing far outweighed the cost of clinical outcomes averted. With mild-severity variants circulating, when removing testing costs from costs averted, symptom-based testing generated a net cost of $990 (range, −$81,579 to $66,281) from the CMS perspective, $1,545 (range, −$199,558 to $102,277) from the third-party payer perspective, $54,765 (range, $23,768 to $90,788) in productivity losses, and $57,155 (range, −$175,790 to $173,348) from the societal perspective.
These significant costs did not return enough positive health effects to make testing cost-effective as measured by the ICER (ratio of the difference in costs over the difference in effectiveness) when using the $50,000 per QALY cost-effectiveness threshold. The ICER exceeded this threshold in 64% of simulation trials from the CMS, 59% of simulation trials from third-party payers, and 85% of simulation trials from the societal perspective. Thus, symptom-based antigen testing was cost-ineffective because net costs were much higher and net health effects were small.
Testing remained cost-ineffective when staff were tested on a weekly basis in addition to symptom-based testing. Table 1 shows data indicating that testing remained cost-ineffective when the frequency of routine testing varied from once every other week and to twice per week. Additionally, testing with the more costly but highly sensitive PCR testing was cost-ineffective (ICERs ≥$1,167,511per QALY).
Table 1. Epidemiologic, Clinical, and Economic Outcomes Averted [Median (range)] by Various Nursing Home Staff COVID-19 Antigen Testing and Isolation Strategies for Different Risks of Nursing Home Staff Acquiring COVID-19 in the Community
Note. Negative net costs are cost savings.
How the value of testing changes with the risk of COVID-19 in the community
Given the changing contagiousness of SARS-CoV-2 variants across winter season, the next set of scenarios explored how the value of testing may change with SARS-CoV-2 contagiousness in the community. We varied the risk of staff acquiring COVID-19 in the community from 10% to 50% over 3 months (0.12%–0.76% per day). However, given the mild severity of the omicron variant, testing and isolation were cost-ineffective, even with higher staff case counts (Table 1). The number of cases averted decreased with increases in the community risk because more cases were brought into the nursing home, resulting in more chances for infection before testing.
How the value of testing changes with face mask use
We varied face mask use compliance from 50% to 90%. Although the number of cases increased with decreased compliance, all testing strategies remained cost-ineffective (median ICER, ≥$83,291 per QALY). For example, when compliance decreased to 70%, symptom-based antigen testing averted 1.0 case (range, −39 to 47) among staff and residents but was cost-ineffective from all perspectives, with ICERs above the $50,000 per QALY threshold in ≥59% of simulation trials. Furthermore, even when staff did not routinely use face masks, all testing strategies remained cost-ineffective, with ICERs above the $50,000 per QALY threshold in ≥55% of trials (eg, median ICER, $121,104 per QALY from the CMS perspective).
How the value of testing changes with vaccination efficacy and coverage
Varying vaccination efficacy against infection (10%–60% for staff and 25%–75% for residents) and vaccination coverage with the bivalent booster (10%–50% for staff and 30%–70% for residents) also did not substantially change the value of testing. The various testing strategies remained cost-ineffective. For example, decreased vaccine efficacy caused cases to rise, but weekly antigen testing averted only 6 cases (range, −17 to 23) among staff and 11 cases (range, −19 to 58) among residents because cases were not identified or isolated quickly enough to substantially reduce spread. More frequent testing was cost-ineffective, with ICERs well above the $50,000 per QALY threshold in ≥71% of simulation trials (median ICER, ≥$1,117,206 per QALY from all perspectives). Similarly, reducing coverage of the bivalent booster increased cases and decreased the number of cases averted (eg, weekly testing averted 7 cases among staff and 14 cases among residents with 10% coverage among staff and 30% among residents). Across a range of vaccine efficacy and coverage estimates, testing remained cost-ineffective because individuals still had protection from the primary series and initial booster vaccines against severe illness.
How the value of testing changes with the severity of clinical outcomes
The severity of clinical outcomes did affect the cost-effectiveness of different testing strategies, such that all strategies became cost-effective when the risk of hospitalization and death was at least 3 times higher than that seen with 2022–2023 omicron variants (Table 2). This risk corresponds to hospitalization and death rates observed earlier in the pandemic with other SARS-CoV-2 variants.
Table 2. Epidemiologic, Clinical, and Economic Outcomes Averted [median (range)] by Various Nursing Home Staff COVID-19 Antigen Testing and Isolation Strategies When Varying the Risk of Severe Clinical Outcomes
Note. Negative net costs are cost savings. Bold values indicate where testing was cost-effective compared to no testing when using a $50,000 per QALY cost-effectiveness threshold. Scenarios assume nursing home staff have a 50% risk of acquiring COVID-19 in the community.
Discussion
Our model results demonstrate that, under winter 2023–2024 omicron variant SARS-CoV-2 conditions, COVID-19 testing and isolation for ≥6 days was cost-ineffective for all testing strategies explored. This remained true when varying COVID-19 contagiousness in the community, face mask compliance, bivalent booster vaccination coverage, and vaccine efficacy against infection. Thus, the costs of testing and isolation far outweighed the cost of clinical outcomes averted. Even when not including isolation costs (ie, productivity losses), testing was cost-ineffective. However, when increasing the severity of clinical outcomes, as seen earlier in the pandemic or if a new or worse variant emerges, testing and isolation became cost-effective.
Our results emphasize that the cost-effectiveness of interventions implemented during a pandemic are highly dependent on clinical outcome severity and other interventions in place. This is important because, as pandemics evolve, greater knowledge about effective prevention measures emerges and enables effective vaccines and personal protective equipment. Thus, reductions in viral virulence, either due to adaptive mutations or human prevention efforts, should be monitored and evaluated over time to ensure that guidance for testing remains beneficial. Monitoring and adjusting becomes increasingly important with multilayered interventions.
Monitoring and re-evaluation become important in at least 3 settings: (1) as severe cases decline with immunity development (through infection or vaccination) and better application of preventative activities; (2) as variants evolve to become more or less deadly; and (3) when pandemic recovery results in a deintensified phase in which vaccinations are updated and preventative activities wane. Reference Kaufman21–Reference Groppe and Fraser23 Our findings suggest that it is necessary to monitor variant severity and resulting clinical outcomes to quickly adapt testing protocols (eg, as severity decreases, testing and isolation strategies become less valuable and may be reduced or stopped). Under such conditions, it is important to determine whether robust vaccination uptake and face mask use is necessary to maintain with decreased severity or if normalization of activities is recommended due to negative trade-offs on socialization and mental health. Adaptiveness may need to become the norm as the pandemic evolves.
This study had several limitations. All models are simplifications of reality and therefore cannot account for every possibility. Reference Lee24 We assumed a fixed staffing ratio and that staff assignments did not change when staff were sent home. In reality, these conditions may result in understaffing, staff interacting with more residents, or the nursing home bringing in additional staff from outside the nursing home, potentially increasing transmission. This situation may increase the value of symptom-based testing if more individuals report symptoms; however, it may be difficult for routine testing to identify cases quickly enough to isolate before transmission occurs. Although we modeled the quarantine of roommates of residents who tested positive, we did not represent contact tracing among other residents and staff when an individual tested positive, which would have increased the number of individuals isolated and the cost of testing.
Under winter 2023–2024 omicron variant conditions, COVID-19 testing and ≥6-day isolation is cost-ineffective. Testing became cost-effective when the severity of clinical outcomes increased. The value of testing depends on outcome severity and other interventions in place. Thus, nursing home administrators and policy makers should regularly monitor and evaluate viral virulence and the value of interventions so that guidance for testing remains beneficial, especially with multilayered interventions.
Supplementary material
To view supplementary material for this article, please visit https://doi.org/10.1017/ice.2024.9
Acknowledgments
Statements in the manuscript do not necessarily represent the official views of, or imply endorsement by, the National Institute of Health, Agency for Healthcare Research and Quality (AHRQ), the US Department of Health and Human Services, the City University of New York (CUNY), or the Pandemic Response Institute (PRI).
Financial support
This work was supported by the AHRQ (grant no. R01HS028165-01), by the National Institute of General Medical Sciences as part of the Models of Infectious Disease Agent Study network (grant nos. R01GM127512 and 3R01GM127512-01A1S1), by the National Science Foundation (proposal no. 2054858), by the NIH Common Fund’s Nutrition for Precision Health, powered by the All of Us Research Program and the National Center for Advancing Translational Sciences of the National Institutes of Health (award no. U54TR004279), and by the CUNY in support of the Pandemic Response Institute (PRI).
Competing interests
All authors report no conflict of interest related to this article.
