INTRODUCTION
Influenza virus is associated with substantial morbidity and mortality worldwide [Reference Monto1]. Households are an important confined setting for influenza transmission [Reference Carrat2–Reference Monto, Koopman and Longini6], and it has been estimated that around a third of all influenza transmission occurs in households [Reference Ferguson3–Reference Kwok, Leung and Riley5]. Recent household studies have investigated the effectiveness of antiviral treatment and prophylaxis [Reference Hayden7–Reference van Boven11], hand hygiene [Reference Cowling12–Reference Simmerman15], face masks [Reference Cowling13–Reference Canini17], and transmissibility of seasonal influenza [Reference Viboud18] and 2009 pandemic influenza A(H1N1) [Reference Cauchemez19–Reference Klick27].
There are two main types of design for household studies that are useful in investigating the efficacy of interventions to prevent household transmission. The first type is a cohort study in which a cohort of initially uninfected households can be recruited and then followed up through periods of influenza activity [Reference Monto, Koopman and Longini6, Reference Monto, Koopman and Bryan28]. This design will be resource intensive if the expected number of households in which an infection occurs is relatively small. Alternatively, households can be enrolled in a study once an influenza infection is identified in one member (an ‘index’ case), and subsequently followed up to observe secondary infections. This design is termed a case-ascertained design [Reference Yang, Longini and Halloran29], and efficient planning of case-ascertained studies was the focus of our first paper in this series [Reference Klick, Leung and Cowling30]. Compared to cohort studies the case-ascertained design may suffer from selection bias, because index cases who present to clinics with symptoms may have more severe illness than influenza virus infections on average, resulting in the introduction of potential selection biases in the assessment of transmission dynamics [Reference Cowling25]. Additionally, difficulty in interpreting data from case-ascertained studies may occur when treatment efficacy decreases with time since symptom onset [Reference Cowling13, Reference Suess14] due to the possibility of infectious contacts being made before the intervention was applied. In both cohort and case-ascertained studies, influenza virus transmission is typically measured via the secondary attack proportion (SAP), defined as the proportion of household contacts that are infected with influenza virus from the index case [Reference Cauchemez19], and sometimes approximated by the proportion of household contacts of an index case that subsequently become infected with influenza (regardless of the presence or absence of direct transmission from the index case) [Reference Lau31].
While some household studies rely entirely on self-reports of symptoms and signs that are associated with acute respiratory illnesses (ARIs) [Reference MacIntyre16–Reference Looker21], generally home visits can be arranged to collect specimens and allow virological confirmation of influenza virus infections [Reference Cowling13, Reference Simmerman15, Reference Papenburg23–Reference Suess26]. Furthermore, specimens for virological confirmation can be collected for all household members at some optimal time after symptom onset in the index case [Reference Cowling13, Reference Klick, Leung and Cowling30] or specimens can be collected only when household contacts report ARI [Reference Monto8, Reference Welliver10, Reference Klick, Leung and Cowling30]. This raises an important question: which combination of study design and paradigm for collection of specimens would be the most powerful option to evaluate and compare the protective effect of interventions? While smaller sample sizes are required in case-ascertained designs to observe an equivalent number of secondary infections compared to a cohort study [Reference Yang, Longini and Halloran29, Reference Klick, Leung and Cowling30], this question has not been previously studied systematically in the literature.
Identifying and selecting an appropriate study design is part of good clinical practice in all clinical research studies. Poorly designed studies may waste precious resources, have inappropriate statistical power and put participants at unnecessary risk and inconvenience [Reference Vollmer and Howard32]. In the present study, we evaluated which study designs make most cost-effective use of resources for maximizing statistical power, in the context of a potential trial of a non-pharmaceutical intervention (NPI).
METHODS
As a basic scenario, we consider planning a study to assess the effectiveness of a NPI study in which we aim to demonstrate that the NPI reduces household transmission compared to a control or minimal intervention. We consider four possible designs for this study:
-
(1) A household cohort study with confirmatory diagnosis of index cases with reverse transcriptase–polymerase chain reaction (RT–PCR) followed by routine collection of specimens of all household members and testing with RT–PCR regardless of reported illness.
-
(2) A household cohort study with confirmatory diagnosis of index cases with RT–PCR followed by collection of specimens and testing with RT–PCR of household members upon report of ARI.
-
(3) A case-ascertained study with routine virological testing of all household members with RT–PCR regardless of reported illness.
-
(4) A case-ascertained study with ARI-triggered collection and testing of specimens of household members.
Here we define ARI as presence of two of the following signs or symptoms: fever (⩾37·8°C), cough, headache, sore throat, or myalgia [Reference Cowling13, Reference Cowling25].
Cohort studies
In these scenarios, we assume that households are recruited in advance of an influenza epidemic. Households are randomly assigned in equal numbers to either an intervention or control group. Household members are instructed to implement the intervention if any household member develops ARI. The aim of the intervention is to reduce household transmission rather than to reduce the risk of infection from the general community. Participating households are encouraged to contact the study team if any member of the household developed symptoms of ARI. Bi-weekly phone calls can also be made to monitor for ARI. As soon as a household reports ARI, a home visit is made to collect respiratory specimens for virological testing. If an index case tested positive for influenza virus infection, either one additional home visit would be made to collect specimens for testing household contacts or a home visit would be made after another household member reported ARI.
Case-ascertained studies
In this scenario, we assume that relatively inexpensive point-of-care rapid tests for influenza are used to identify influenza virus infection in individuals with ARI presenting to a study clinic [Reference Hurt33]. Once a case is identified, the index case and his/her household are recruited for inclusion in the study. For those index cases with a positive rapid test result, an initial home visit is conducted as soon as possible to collect respiratory specimens from all household contacts for laboratory testing to determine whether there were any co-primary cases. If a co-primary case was identified, that household would not be enrolled in the study [Reference Cowling13, Reference Sikora34]. We assume that a home visit confirms that the remaining household members are negative for influenza and also that the household can be enrolled. It is assumed that the intervention takes place in the enrolled household between 12 and 60 h after symptom onset of the index case. Either an additional home visit 6 days after symptom onset in the index case is made [Reference Klick, Leung and Cowling30] or a home visit is conducted immediately after a report of onset of ARI in a household contact to collect respiratory specimens for virological confirmation of potential secondary infections.
Sources of data
Some basic parameters are required for the simulation characterizing influenza transmission in households (Table 1). We assume that in addition to the index case there are three additional household members. We also assume that within-household tertiary transmission is relatively rare and has limited effect on the estimation of the SAP. Estimation of the SAP using the generalized estimating equation (GEE) approach described below accounts for clustering within households but does not explicitly model chains of transmission [Reference Halloran, Préziosi and Chu35]. While our model adopts a single value for an overall mean SAP, we allow the actual SAP to vary stochastically in households. We also assume there is some small correlation between household SAP and illness severity. The index cases with greater illness severity are more likely to be enrolled in case-ascertained studies as their symptoms must be severe enough to require medical attention.
Table 1. Baseline epidemiological parameters
ARI, Acute respiratory illness; CA, case-ascertained; NPI, non-pharmaceutical intervention; PCR, polymerase chain reaction; SAP, secondary attack proportion.
Costs must also be considered, since an optimal design must involve a trade-off between the power and the cost required for a certain sample size with the number of follow-up visits. We specified the recruitment, enrolment and testing costs per household for case-ascertained and cohort studies based on previous household studies conducted in Hong Kong.
Statistical analysis
For both cohort and case-ascertained study design variants, we estimated the power of each potential variant to detect a NPI effect in terms of a reduction in the estimated SAP in the intervention vs. the control group, expressed as an odds ratio. A logistic regression model with GEE [Reference Halloran, Préziosi and Chu35, Reference Diggle36] accounting for within-household correlation was fitted to the simulated data. The statistical power was estimated as the number of simulated datasets in which the intervention effect was identified at a significance level of P⩽0·05.
Due to the nonlinearities in the transmission dynamics of influenza, we used a simulation approach to compare alternative study design variants [Reference Moineddin, Matheson and Glazier37]. For each study design variant we used a Monte Carlo approach to simulate a set of 2500 datasets. The power of each variant was evaluated by statistical analysis of the set of 2500 simulated datasets and compared across design variants. We chose 2500 iterations to ensure the Monte Carlo error was ⩽0·01 [Reference Robert and Casella38]. Further technical details are provided in the online Supplementary Appendix.
Sensitivity analyses
To examine the impact of variations in key model parameters on the optimal study design, we performed a set of sensitivity analysis varying several model parameters. The effect of changes in the community probability of infection (CPI) was examined, because it influences the rate at which a susceptible individual acquires infection from the community during the influenza epidemic [Reference Longini39]. Another sensitivity analysis examined the optimal study variants with shorter [Reference Ferguson3, Reference Cauchemez19, Reference White40] or longer [Reference Cowling41] serial intervals than assumed in the baseline scenario. If the serial interval was shorter, this could allow more infectious contacts before the intervention is implemented. Finally, we examined the sensitivity of enrolment cost per household on the optimality of case-ascertained and cohort studies.
RESULTS
The results of our analysis using baseline parameter values are shown in Figure 1. The number of households that can be recruited per arm given a fixed fieldwork budget is given in Figure 2. The case-ascertained design with ARI-triggered collection of samples had the highest power. This was followed by a cohort study with ARI collection of samples and a case-ascertained study with routine collection of virological specimens regardless of reported symptoms. The least powerful design was found to be a cohort study with routine collection of specimens once an index case was identified regardless of reported symptoms in the household contacts. Budgets of US$1.7 million (1152 households per arm), US$1.9 million (1659 households per arm), US$2.1 million (1104 households per arm), and US$2.3 million (1557 households per arm) were required to achieve a study of 80% power for a case-ascertained study with ARI trigger, a cohort study with ARI trigger, a case-ascertained study with routine testing, and a cohort study with routine testing, respectively.
Fig. 1. Statistical power of competing study designs as a function of study budget. ARI, Acute respiratory illness; CA, case-ascertained; PCR, polymerase chain reaction.
Fig. 2. Number of households per arm that can be enrolled for competing study designs as a function of study budget. ARI, Acute respiratory illness; CA, case-ascertained; PCR, polymerase chain reaction.
Sensitivity analyses
The total budget needed to achieve 80% statistical power from sensitivity analyses varying key parameter is shown in Figure 3. Sensitivity analyses showed that either case-ascertained or cohort designs could be the most powerful design depending on parameter values. The power of cohort designs was sensitive to the CPI (Fig. 3, Appendix Fig. S1). A higher CPI causes cohort studies to have higher power. The power of case-ascertained designs was highly sensitive to the serial interval (Fig. 3, Appendix Fig. S2). As was intuitively expected, shorter serial intervals led the intervention to be less effective. The power of both case-ascertained and cohort studies were sensitive to enrolment costs, leading either case-ascertained or cohort studies to be most powerful depending on parameter values (Fig. 3, Appendix Figs S3, S4).
Fig. 3. Total budget necessary to achieve 80% statistical power. Diamonds represent estimates from sensitivity analyses while squares represent estimates from the main analysis. (a) Sensitivity analysis showing the effect of changes to the community probability of infection (CPI). Lower diamonds represent results with a low CPI of 0·10 and upper diamonds represent results with a high CPI of 0·30. Changes in CPI do not affect the results for case-ascertainted studies. (b) Sensitivity analysis showing the effect of changes to the serial interval of influenza. Lower diamonds represent the model results with a shorter serial interval (mean 2·6) and upper diamonds represent the model results with a longer serial interval (mean 3·6). (c) Sensitivity analysis showing the effect of changes to the enrolment cost of households. Lower diamonds represent the model results reducing enrolment costs by 33% and upper diamonds represent increasing enrolment costs by 33%. ARI, Acute respiratory illness; CA, case-ascertained; PCR, polymerase chain reaction.
DISCUSSION
Careful consideration is required when planning household transmission studies of influenza. Our results illustrate that case-ascertained designs are the most resource efficient design for testing NPI efficacy. However, this conclusion was sensitive to the costs of enrolment, cost of laboratory methods, and the risk of influenza in the community. Since all cohort studies are very sensitive to CPI, and because this probability markedly varies throughout the course of influenza seasons and cannot be accurately predicted in advance, case-ascertained designs may be considered to be less vulnerable to epidemic dynamics than cohort designs. As we found in the first paper in this series, using ARI to trigger home visits can often be a cost-effective strategy in both case-ascertained and cohort designs. However, it should be noted that use of ARI trigger may miss some asymptomatic and subclinical infections [Reference Lau42] and also that conducting routine home visits may encourage intervention adherence for control purposes.
Several practical considerations may help in the choice between a case-ascertained and cohort design. Testing NPIs using a case-ascertained design may be practically considered as the best-case scenario, because households have just been recruited into the study and taught how to use an intervention. Therefore, it might be expected that adherence to the intervention within households recruited for a case-ascertained study would be higher than it would be in the general population during an influenza epidemic or pandemic. However, case-ascertained studies will always suffer from some truncation problems because there will inevitably be a delay between symptom onset in the index case and the implementation of an intervention. Therefore some secondary transmission may occur before the start of the intervention and this has been noted in several case-ascertained studies [Reference Cowling13, Reference Suess14]. Cohort studies may permit more representative estimates of the efficacy of NPIs in the general population, while it should be borne in mind that adherence may be lower than a case-ascertained design due to waning compliance over time. It is also notable that case-ascertained study designs may be a good choice for assessing secondary aims such as the duration and severity of symptoms and viral shedding [Reference Lau42], while cohort designs could also address secondary aims such as annual incidence and risk factors for infection and illness.
Our study has some limitations when considering the application of the results to planning actual studies. First, our estimates of the costs of enrolment and testing are specific, based on our experience in Hong Kong. These may differ in other countries in a manner that would affect the results, and thus, the optimality should ideally be calibrated to other settings. Second, we did not consider self-swabbing of study participants as a strategy for confirmation of influenza virus infections. This strategy has been proposed as a cost-saving and convenient method for collection of respiratory specimens for testing [Reference Elliot43–Reference Ip46]. While early evidence is promising and suggests that sensitivity for self-swabbing may be nearly as high as when swabs are collected by trained personnel [Reference Elliot43–Reference Ip46], questions remain about participant compliance and overall cost savings of this strategy. Third, we did not consider identification of influenza using serological methods due to uncertainties relating to the costs and accuracy relative to RT–PCR confirmation. However, household studies using serological testing have been previously used to test NPI efficacy [Reference Longini and Monto47]. Fourth, we assumed that in case-ascertained studies that index cases presenting to clinics less than 48 h after symptom onset would be eligible to participate. If this time could be shortened, then the cost efficiency of case-ascertained studies would be elevated. Fifth, it is difficult to gauge adherence in cohort studies and the study protocol could affect adherence. Finally, several papers [Reference Yang, Longini and Halloran29, Reference Halloran48] have noted that direct randomization of household members rather than cluster randomization of households is more powerful and can allow for the estimation of the efficacy of an intervention in both reducing the risk of infection for susceptible household members and reducing the infectiousness of an infected household member. However, for NPIs the intervention cannot usually be blinded from subjects, and cluster randomization may be more feasible and acceptable for implementation.
Despite the sensitivity of our models to the choice of parameters and potential limitations, our results serve as useful guidelines for researchers in planning future household studies of influenza interventions. Our findings suggest that none of the designs performed particularly poorly and that secondary considerations might warrant the use of designs not considered optimal in terms of statistical power.
SUPPLEMENTARY MATERIAL
For supplementary material accompanying this paper visit http://dx.doi.org/10.1017/S0950268813001623.
ACKNOWLEDGEMENTS
This work received financial support from the Research Fund for the Control of Infectious Disease, Food and Health Bureau, Government of the Hong Kong SAR (grant no. HK-10-04-05), the Area of Excellence Scheme of the Hong Kong University Grants Committee (grant no. AoE/M-12/06) and the Harvard Center for Communicable Disease Dynamics from the National Institute of General Medical Sciences (grant no. 1 U54 GM088558). B.K. is supported by a Hong Kong Ph.D. Fellowship from the Research Grants Council, Hong Kong. H.N. received financial support from the JST PRESTO programme. The funding bodies were not involved in the collection, analysis and interpretation of data, the writing of the manuscript, or the decision to submit for publication. The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Institute of General Medical Sciences or the National Institutes of Health.
DECLARATION OF INTEREST
B.J.C. has received study funding from MedImmune Inc., and consults for Crucell NV.
