By the time the World Health Organization (WHO) declared coronavirus disease (COVID-19) a pandemic on March 11, 2020, 113 countries had confirmed cases, with 80 955 cases reported in China and 696 in the United States. 1,2 By the end of that month, 5% of all worldwide cases 3 and 1 in 3 of US cases and deaths were in New York City. 4,5 This created a real concern that hospitals might suffer a shortfall of resources and need to ration ventilators. Reference Cha, Bernstein and Sellers6–Reference Rosenthal and Goldstein9 Developed in 2007 and updated in 2015 to address such a scenario, the New York State Department of Health along with New York State Task Force on Life and the Law released the New York State Ventilator Allocation Guidelines (NYSVAG). 10 This document draws upon a community’s duty to care, resource stewardship, transparency, and distributive justice to save the most lives. Briefly, this unvalidated strategy heavily utilizes the Sequential Organ Failure Assessment (SOFA), a summative 0-24 score of organ dysfunction across neurologic, pulmonary, cardiovascular, hematologic, hepatobiliary, and renal domains, to prioritize (colored red for highest or yellow for intermediate) and potentially exclude patients from a ventilator (colored blue). Patients triaged and provided a ventilator receive 1 for a pre-specified duration of time. After the ventilator time trial, their SOFA scores are recalculated, and patients are re-triaged to maintain their ventilators or have them reallocated to other patients. NYSVAG are very similar to other available crisis standard of care guidelines, Reference White and Lo11–Reference Piscitello, Kapania and Miller15 62.1% of which utilize a SOFA score. Moreover, and similar to NYSVAG, 55.6% exclude patients from a ventilator for a SOFA score > 11, and 65.5% use the same ventilator time trial intervals (2 days, 5 days, and every subsequent 2 days). Reference Fiest, Krewulak and Plotnikoff14,Reference Piscitello, Kapania and Miller15 With only 1 documented case of ventilator rationing by NYSVAG exclusionary critiera in New York City during the original surge, Reference Fins16 and with Alaska 17 and Idaho 18 having both activated regional crisis standards of care that explicitly permit ventilator rationing (Alaska uses verbatim SOFA cutoffs as NYSVAG and Idaho use a similar SOFA and time trial criteria as NYSVAG), it remains crucial to understand how these guidelines might perform.
Objectives
It remains uncertain how such ventilator allocation guidelines would perform if implemented. This study simulates how specific SOFA cutoffs for resource prioritization and the ventilator time trials, used in NYSVAG and other crisis standards of care documents, might affect ventilator allocation and patient survival in a COVID-19 cohort during the pandemic. To address this research question, the relevant portions NYSVAG were simulated on every patient independently to one another and independent to the number of available ventilators, thereby aligning temporally discordant patient hospitalizations. This design can reveal important guideline inefficiencies that might potentiate mortality and remain obscure when incorporating a fixed ventilator supply with variable or low ventilator strain. Moreover, this analysis is helpful to evaluate how explicit triage criteria (SOFA cutoffs and ventilator time trials) would perform in an isolated manner that is independent to the resources available at a facility. Such an evaluation can determine whether a resource allocation strategy may be effective given the trajectory of patient survival or whether it might reallocate resources away from patients who would otherwise survive. Additionally, a retrospective analysis where each patient received a ventilator provides a unique control group to simulate resource allocation protocols because their survival with a ventilator is known. The results of this analysis remain germane to developing efficient crisis standard of care procedures and could provide reassurance of existing ventilator allocation guidelines or suggest the need for further revisions.
Methods
Adult patient charts were retrospectively reviewed from a single academic hospital system encompassing 1 tertiary care referral center and 2 community hospitals from 3/1/2020–7/1/2020. Patients were included if they were SARS-CoV-2 positive and were invasively mechanically ventilated. Components of the SOFA score were obtained for each patient daily as close to 10:00 am as possible. Missing components of the SOFA score were presumed normal. Reference Knaus, Wagner and Draper19,Reference Wunsch, Hill and Bosch20 If there was no reported SOFA score on the day of intubation, the SOFA score from the next day was used instead for the day of intubation (and for relevant comparisons). Every patient received a ventilator as medically indicated, and ventilator allocation or reallocation per NYSVAG only occurred in a virtual simulation.
NYSVAG were interpreted for this study in the following manner: Step 1, the application of exclusionary criteria, was deferred because the number of such patients was negligible (Supplement Figure 1). Reference Wunsch, Hill and Bosch20 Steps 2 and 3, the allocation and potential reallocation of ventilators respectively, were simulated on each patient as described in NYSVAG (Table 1; Supplement Figures 2, 3a-b). Patients categorized as Red received the highest priority for a ventilator, followed by those patients classified as Yellow. Patients classified as Blue were ineligible for a ventilator.
a Three gaps in NYSVAG that were addressed as follows: (1) Patients with a SOFA score of 7 on the day of intubation were categorized as Red; (2) individuals with a SOFA score greater than 7 on any assessment after 5 days of mechanical ventilation were ineligible for a ventilator; and (3) patients ventilated for 7 or more days who continued to show a SOFA score ≤ 7 and decreasing were placed in the Yellow category.
NYSVAG were applied to the cohort in the following simulation (see Supplemental Methods for additional details): Every patient’s SOFA score was calculated on the day of intubation, Day 2 of mechanical ventilation, Day 5, and every subsequent 2 days of mechanical ventilation. Any patient who was characterized as Red or Yellow would receive (or maintain) a ventilator; however, any patient characterized as Blue would have her or his ventilator withheld (or reallocated) on the day that patient was characterized as ineligible for a ventilator. This design was chosen because this analysis can evaluate a central component of the guidelines (SOFA score cutoffs and ventilator time trials) independent to the number of available ventilators, patient throughput, ventilator strain, or impact from randomization. Patients who had their ventilator withheld or reallocated were simulated to expire as modeled in other simulation studies withholding or withdrawing life-sustaining treatment. Reference Kanter21,Reference Bhavani, Luo, Miller, Sanchez-Pinto, Han, Mao, Sandıkçı, Peek, Coopersmith, Michelson and Parker22 Patients were also removed from the simulation if they expired, were extubated, or placed on extracorporeal membrane oxygenation (ECMO) between assessment days.
Outcome measurements include both observed (what occurred in the real world) and simulated (what occurred hypothetically) endpoints. Because every patient in our cohort received a ventilator in the real world, their performance with respect to survival to hospital discharge with continued ventilation is known (referenced as an observed outcome). The primary observed endpoints include death or survival to discharge. The primary simulated endpoints include whether the patient’s ventilator was simulated to be withheld or reallocated on the day of intubation, 2, 5, 7, 9, or 11 days post-intubation (Figure 1). Combined simulated-observed endpoints include whether the patient had her or his ventilator reallocated (simulated) but was observed to survive to hospital discharge (observed, how the patient would have performed if provided with a ventilator). Comorbidities were queried from the ICD-10 problem list.
Missing data were amended in 2 subsequent simulations. First, all Glasgow Coma Scale (GCS) SOFA subscores were normalized (GCS-nl). Second, missing pulmonary SOFA subscores were supplemented by the modified SOFA pulmonary category (mSOFA-FiO2). As oxygen saturation was not gathered and FiO2 was, pulmonary mSOFA scores were computed with a presumptive oxygen saturation of 85% or better. A fourth simulation delayed the Day 2 reassessment to Day 5, the Day 5 reassessment to Day 7, and the subsequent 2-day reassessments to start on Day 9 (MVD 2to5).
Categorical data were analyzed with the chi-square test using the 95% confidence interval (CI). Continuous variables were compared with a 2-tailed t-test using a 5% significance level.
This study was approved by the NYU Grossman School of Medicine Institutional Review Board (study #i20-01653). No treatment, ventilator allocation, or reallocation decisions were made for any patient. Patients were exempt from informed consent.
Results
Observed Results
COVID-19 patients, 884 in total, were intubated between 3/1/2020 and 7/1/2020 and included in the simulation. The average age of the cohort was 63 years, predominately male (70.0%), with varied racial ethnicities (Table 2). Comorbidity information was only available for 50% the cohort (Supplement Table 1). The median (IQR) duration of mechanical ventilation for the cohort was 10.2 (3.74, 23.73) days, with 41.0% survival to discharge.
Simulation Results
NYSVAG were applied to the observed patient SOFA scores in a simulation shown in Figure 2 and Table 3. Table 4 depicts how simulated patients, on each day of reassessment, were triaged as ineligible to receive or maintain a ventilator (Blue) and how many were observed to survive to discharge.
Numerical representation of Figure 2 depicting the simulated application of NYSVAG on a cohort of critically ill COVID-19 patients.
a Survival to hospital discharge is an observed outcome.
b Characterized as ineligible for a ventilator (Blue) and simulated for ventilator removal/withholding is a simulated outcome.
c Combined simulated-observed outcome; those who were classified as ineligible for a ventilator (Blue) in the simulation but were also observed to survive to hospital discharge when they were provided a ventilator in the real world.
Etiology of patients classified as Blue and triaged as ineligible for a ventilator on the specified reassessment day of mechanical ventilation.
Step 2: Assessment at intubation
In total, 884 patients underwent Step 2 of NYSVAG—prioritization and allocation of a ventilator. Average SOFA on the day of triage for mechanical ventilation was 7.4 ± 3.3; 478 (54.1%) individuals qualified as Red, the highest priority for a ventilator, 285 (32.2%) as Yellow or intermediate priority for a ventilator, and 121 (13.7%) as Blue or ineligible for a ventilator. Of the 121 individuals triaged as Blue and not provided with a ventilator in the simulation, 31 (25.6%) were observed to survive to discharge. Of the 763 patients who were initially prioritized to receive a ventilator (Red or Yellow), 249 (32.6%) would later be triaged in the simulation to have their ventilator reallocated (Blue) but would have otherwise been observed to survive to discharge (observed outcome).
Step 3: Day two reassessment
From the day of intubation to 2 days post-intubation, 41 individuals had expired, 35 had been extubated, and 10 had been placed onto ECMO. As such, 678 patients were reassessed on post-intubation Day 2. Of this group, 223 (32.9%) were characterized as Red and 50 (7.4%) as Yellow. Per NYSVAG, for those being reassessed after 2 days of mechanical ventilation, 404 individuals (59.7%) would have had their ventilator reallocated, 136 (33.7%) of whom were observed to survive to discharge. Among these 404 individuals, both the 90 individuals (22.3% of all reassessed as Blue on this reassessment, 47.8% observed survival) who had a worsening SOFA score less than 7 and the 204 individuals (50.5%, 35.3% survival) who had a worsening SOFA score of 8-11 were more statistically likely to be discharged alive if provided a ventilator than those 110 individuals (27.2%, 19.1% survival) who had a SOFA score greater than 11 (P < 0.001 and P = 0.003, respectively). Individuals characterized as Blue with a worsening SOFA score less than 7 were also more likely to survive to hospital discharge with a ventilator than those with a SOFA score of 8-11 (P = 0.04).
Step 3: Day five reassessment
Of the 678 patients from Day 2, 24 individuals were extubated, 16 expired, 7 were placed onto ECMO, and 405 were extubated due to the ventilator reallocation that occurred because of the Day 2 reassessment. As such, 226 patients were present to be reassessed after 5 days of mechanical ventilation. At this time, 25 individuals (11.1%) were triaged as Red and 39 (17.3%) as Yellow; 162 patients (71.7%) were triaged as Blue and were simulated to have their ventilator reallocated, 75 (46.3%) of whom were observed to survive to discharge. Among these 162 individuals, both the 88 individuals (54.3% of all reassessed as Blue, 55.7% observed survival) who had a worsening SOFA score less than 7 and the 61 individuals (37.7%, 41.0% survival) who had a SOFA score of 8-11 were more statistically likely to be discharged alive if provided with a ventilator than those 13 individuals (8.0%, 7.7% survival) who had a SOFA score greater than 11 (P = 0.001 and P = 0.02, respectively).
Step 3: Day seven reassessment
Of the 226 patients from Day 5, 5 individuals were extubated, 3 expired, and 162 were extubated due to the ventilator reallocation on the Day 5 Reassessment. As such, 56 patients were present to be reassessed after 7 days of mechanical ventilation. At this time, 7 individuals were prioritized for a ventilator. 49 patients (87.5%) were characterized as Blue and simulated to have their ventilator reallocated, 34 (69.4%) of whom were observed to survive to discharge. Amongst these 49 individuals, the 42 individuals (85.7% of all reassessed as Blue, 78.6% observed survival) who had a worsening SOFA score less than 7 were more statistically likely to be discharged alive if provided with a ventilator than and the 7 individuals (14.3%, 14.3% survival) who had a SOFA score of 8-11 (P < 0.001).
After 11 days of simulation, all 884 patients would have either been extubated and survived to hospital discharge, expired, had their ventilator reallocated, or placed onto ECMO; 742 patients would have had their ventilator reallocated (83.9% of the original cohort), 280 (37.7%) of whom were observed to survive to discharge if provided with a ventilator. Of the total 360 patients observed to survive to discharge, only 65 (18.1%) would have survived to discharge without having their ventilator reallocated in the simulation. There was no statistically significant ventilator reallocation by age, gender, race, or ethnicity at any day of assessment, among all who had their ventilator reallocated, or among those who were simulated to survive.
Missing data and additional analyses
Our cohort required 1852 patient-SOFA-days (a SOFA score for each day a ventilated patient was assessed). The majority of missing information was on the day of intubation, with 100 patients having no SOFA score on the day of intubation (Supplement Table 2). Additional modeling with adjusted or supplemented SOFA scores, as well as deferring NYSVAG Day 2 reassessment to Day 5, is shown in Figure 3. We failed to find a significant difference in simulated surviving individuals when normalizing the GCS or supplementing pulmonary SOFA subscores (72 GCS-nl versus 65; see Figure 3). A delay in the Day 2 reassessment to Day 5 (MVD 2to5) resulted in a 49.2% increase in simulated survival to discharge within NYSVAG criteria.
Discussion
Our findings simulate the performance of common triage criteria (SOFA cutoffs and ventilator time trials) on a large cohort of critically ill COVID-19 patients. Ventilator reallocation was simulated to occur for 83.9% (742/884) of the cohort, 37.7% (280/742) of whom would have survived to discharge if provided with a ventilator. Additionally, only 18% (65/362) of all individuals who were observed to survive to discharge were also simulated to survive to discharge within the simulated NYSVAG criteria. This raises a significant concern that under this ventilator allocation protocol, ventilators might be taken away from patients who would otherwise survive. This concern would be assuaged if the subsequent patients who received the reallocated ventilators would have fared much better, thereby improving overall group survival. This would satisfy the NYSVAG utilitarian goal of doing the most good for the most people, commonly interpreted as saving the most lives. 10
Others have articulated that any redistribution of lifesaving resources must be fair, transparent, and promote equity among vulnerable and marginalized populations. Reference Ashana, Anesi, Liu, Escobar, Chesley, Eneanya, Weissman, Miller, Harhay and Halpern23–Reference Cleveland Manchanda, Couillard and Sivashanker26 It is therefore encouraging that any ventilator distribution per NYSVAG was not statistically significant among any age, gender, ethnic, or racial groups.
Our results contrast with that reported by Wunsch et al., a similar retrospective simulation of applying NYSVAG to 40,439 intubated critically ill patients from 2014-2015. Reference Wunsch, Hill and Bosch20 Our cohort of COVID-19 patients was sicker than their mixed medical/surgical population (initial COVID-19 SOFA of 7.4 ± 3.3 compared to 4.5 ± 3.7), which may account for the larger proportion triaged as ineligible for a ventilator (Blue, 13.7% vs 8.9%). On the second or fifth reassessment day, only 4.7% or 28.3% of their cohort was classified as ineligible for a ventilator (Blue), respectively. This contrasts concerningly to the 59.7% and 71.7% simulated in our cohort of COVID-19 patients. Moreover, their median (IQR) duration of mechanical ventilation was 1.94 (0.7, 5.1) days, whereas our COVID-19 patient cohort required a median of 10.2 (3.74, 23.73) days. COVID-19 patients do not recover on a similar trajectory to other critical respiratory illnesses, including influenza, and allocation guidelines may need to extend their reassessment intervals. Determination of the appropriate length of time for a trial of mechanical ventilation can be difficult since the physiologic trajectory of the disease—even with optimal treatment—may be unknown or variable. Too short a trial of ventilation and patients may have an inadequate length of time to demonstrate a clinical benefit. Short of objective evidence that a patient is failing therapy, premature reassessment that results in the removal of resources that might otherwise later benefit the patient may promote rapid cycling between patients irrespective of prognosis and worsen overall mortality. Too long between reassessments might lead to disproportionate resource utilization without optimizing overall survival.
Our results may signal that the recommended length of time for a ventilator time trial was too short to demonstrate improvement for COVID-19 patients. This is supported by the observation that ventilator reallocation would have occurred for over half of the simulated remaining patients on or after 2 days of mechanical ventilation, of which over one third were observed to survive to discharge (see Table 3). It is also corroborated by the 49.2% increase in patients who would survive to discharge if the second day of assessment of ventilator allocation was delayed to Day 5 (see Figure 3). This may suggest that the appropriate length of a time trial should be extended for COVID-19 patients compared to other typical causes of respiratory failure.
Potential Limitations
This study has several potential limitations. Although this simulation is not based on a fixed supply of ventilators and includes an unrealistic dynamic number of available ventilators, it provides an important analysis that best represents the true performance of commonly used SOFA cutoffs and reassessment intervals with respect to patient mortality. This design is crucial because guideline underperformance might be obscured under circumstances of low ventilator strain where the majority of patients are provided with a ventilator regardless of their eligibility. A true understanding of which patients are classified as ineligible for lifesaving resources and their chances of survival remains paramount to understanding how guidelines might perform with increasing resource strain. This cohort study also facilitates a unique opportunity to understand how many patients would have had their ventilator withheld (simulated) but would have otherwise survived if a ventilator was provided (because they were provided with a ventilator in the real world). Our simulation demonstrates the important concern that, according to the NYSVAG criteria studied, many patients who may have had their ventilators withdrawn would have subsequently survived. This concern would be mitigated, and NYSVAG indeed validated, if subsequent patients receiving reallocated ventilators had a higher rate of survival. We acknowledge that our study design cannot explicitly assess the mortality of subsequent patients receiving reallocated ventilators. However, our results suggest that patients ventilated for 5 or more days and selected to have their ventilator reallocated may have a higher chance of surviving to discharge with a ventilator (46.3-75.0%) than subsequent patients receiving their reallocated ventilator (43.4%) (see Table 3). This conjecture, if substantiated, may have significant implications as to how new patients should be triaged in the setting of existing ventilated patients. The survival of patients receiving reallocated ventilators must be explicitly studied before changes to existing protocols are proposed and adopted.
A simulation including a fixed number of available ventilators and patient throughput incorporating ventilator strain will produce different results and allow for the explicit study of patients receiving reallocated ventilators. In such a scenario, however, it remains concerning that the majority of patients classified as Blue on or after 5 days of mechanical ventilation had a SOFA score of less than 7. Randomization may not maximize patient survival if it occurs among all Blue individuals and does not prioritize subcategories that are more likely to survive (see Table 4). A triage officer or committee may provide a more sensitive selection process; however, NYSVAG only permits randomization within a priority color. 10 (p. 68) Further subcategorization of all patients classified as ineligible for a ventilator should be considered in resource allocation protocols. Subsequent studies incorporating a fixed number of available ventilators, real patient throughput, and the comparison of alternative triage strategies such as first-come first-served should be simulated. Bhavani et al. performed a limited simulation; however, they do not incorporate patient throughput or dynamic real-world surge strain. Reference Bhavani, Luo, Miller, Sanchez-Pinto, Han, Mao, Sandıkçı, Peek, Coopersmith, Michelson and Parker22
We believe our results are broadly generalizable. While NYSVAG was designed for an influenza pandemic, the NYSVAG Q&A approve its application for a broader respiratory crisis. 27 Although we analyzed limited aspects of NYSVAG, these elements are similar to several other resource allocation policies. Reference White and Lo11–Reference Piscitello, Kapania and Miller15 We believe our sizeable cohort is generalizable to other COVID-19 patients as our SOFA score and PaO2/FiO2 at intubation Reference Cummings, Baldwin and Abrams28–Reference Grasselli, Zangrillo and Zanella32 and duration of mechanical ventilation Reference Cummings, Baldwin and Abrams28,Reference Ziehr, Alladina and Petri29 are consistent with other reports. Our high mortality is consistent with other local reports during the study period Reference Petrilli, Jones and Yang33–Reference Santos, Levy and Gong35 and likely related to the unprecedented surge and associated ICU strain that would be consistent in communities activating crisis standards of care. Reference Toth, Tatem and Hosseinipour34 . Our NYSVAG simulation results are additionally similar to a 200-patient mixed COVID-19 cohort from a comparable local institution. Reference Santos, Levy and Gong35 While all ventilated patients would participate in the same resource allocation policy, non-COVID-19 patients were excluded from this simulation because they were marginally present during our institutions’ surge.
There remain specific limitations related to the general dependence on SOFA scores for reallocation policies. While an initial report suggested that SOFA scores could predict mortality among COVID-19 patients, Reference Zhou, Yu and Du36 subsequent studies have not been as supportive. Reference Fiest, Krewulak and Plotnikoff14,Reference Raschke, Agarwal and Rangan37,Reference Shahpori, Stelfox and Doig38 Despite such shortcomings, using an objective score to prioritize resource allocation with the intent of saving the most lives achieves impartiality, preserves fairness toward patients, promotes transparency with the community, maintains reproducibility between providers, eliminates the encroachment of bias or prejudices by decision-makers, and may reduce provider distress about ventilator allocation decisions. Such scores may require further augmentation to strive for equity and not worsen existing health disparities, Reference Ashana, Anesi, Liu, Escobar, Chesley, Eneanya, Weissman, Miller, Harhay and Halpern23,Reference White and Lo39 or to better reflect community values. Reference Biddison, Gwon and Schoch-Spana40 While many critically ill COVID-19 patients are at risk of multiple organ sequelae, some remain “single organ” dysfunction with respiratory failure, whereupon SOFA scores may not be the most accurate reflection of the severity of their illness.
Dependence on SOFA scores can be additionally problematic as COVID-19 patients often require prone positioning, which can complicate the interpretation of the oxygenation component, or heavy sedation or paralytics, which can interfere with the neurological assessment. In our simulation, GCS scores were normalized when they were not documented, which was overwhelmingly common and may be related to patients’ clinical contraindications for interruption of sedation or paralytics versus incomplete documentation. The frequency of missing data in Supplemental Table 2 underscores this point and may only be partially mitigated by using the mSOFA score. Our normalization of missing information underestimates a patient’s true SOFA score and therefore may underestimate the frequency of ventilator reallocation; 11.3% of our cohort was intubated without any SOFA information, possibly suggesting that first responders may need to make triage decisions with incomplete information.
Conclusion
States across the nation continue to battle against surges in the COVID-19 pandemic. Federal, state, and local governments should continue to study and develop their disaster preparedness plans. Our results suggest that if NYS Ventilator Allocation Guidelines were implemented, patients might have their ventilator reallocated when they might otherwise survive. Further studies should determine whether subsequent patients receiving reallocated ventilators would have a higher survival than the original cohort before changes to the guidelines are enacted.
Supplementary Material
To view supplementary material for this article, please visit https://doi.org/10.1017/dmp.2022.154
Acknowledgments
We are grateful to Kelly Paluch for her statistical contributions.
Author Contributions
All authors contributed to all aspects except MEN, who was not involved in the study design.
The views expressed in this article are those of the authors and do not necessarily reflect the position or policy of the Department of Veterans Affairs or the United States Government.
Funding Statement
This research is unfunded and approved by the NYU Grossman School of Medicine Office of Science and Research Institutional Review Board i20-01653. This project was presented as an abstract during the American Thoracic Society 2021.
Conflict(s) of Interest
The authors have no conflicts of interest.