INTRODUCTION
Acute lower respiratory tract infections (LRI) are a leading cause of death in the USA, and most deaths occur in persons aged >65 years [Reference Heron and Smith1]. In adults, the incidence of these infections increases with age: almost 1 million episodes occur in persons aged ⩾65 years and about 1/20 persons aged ⩾85 years experience an episode of community-acquired pneumonia (CAP) each year [Reference Jackson2]. Studies of the epidemiology of LRI and CAP are complicated by the challenges in identifying cases and the lack of standardized case definitions. Several previous studies from industrialized countries have attempted to estimate the incidence of CAP [Reference Jackson2–Reference Gable10]. These studies used various case definitions and a variety of case detection methods including surveillance of hospital admissions, review of administrative data from a group health plan, and community-based surveillance of medical practices. These approaches have a number of limitations including the fact that many cases of CAP are diagnosed and treated in the outpatient setting and diagnostic coding is not always accurate [Reference Jackson2].
According to standards set by the American Thoracic Society and the Infectious Disease Society of America, chest radiography is required to confirm the diagnosis of pneumonia and is more sensitive and specific than clinical findings alone [Reference Foy11]. Pneumonia case definitions that depend on clinical features alone are subject to several limitations, such as inter-observer variability in the identification of clinical signs and the transient nature of some clinical findings [Reference Jackson2]. The clinical diagnosis of pneumonia often does not correspond to radiographic findings [Reference Jackson2, Reference Mandell12]. Case definitions based on radiographic findings are also problematic because the interpretation of chest radiographs (CXRs) is subjective and subject to inter-observer variability [Reference Mandell12–Reference Young and Marrie17]. The World Health Organization (WHO) has recently published a methodology for standardized interpretation of paediatric CXRs for use in vaccine trials with pneumonia prevention endpoints [Reference Cherian18]. Ideally, standard radiographic endpoints will enable the comparison of results across different vaccine trials and epidemiological studies. In preparation for a possible adult pneumococcal vaccine trial, we carried out a pilot study in White Mountain Apache adults, a population at high risk for invasive pneumococcal disease [Reference Cortese19]. Our objectives were to test an adaptation (for adults) of the WHO paediatric CXR interpretation system and to estimate the incidence of CAP in adults in this population.
METHODS
Setting
The White Mountain Apache reservation is located in a rural area of Arizona in the southwestern United States. Health care, including radiographic services, is provided at no cost to patients for Native Americans living on or near the reservation by the Indian Health Service (IHS) hospital located in Whiteriver. The only other source of radiography in the region is Summit Health Care Regional Medical Center (SRMC), a private hospital located about 10 miles from the White Mountain Apache reservation. About 14 000 White Mountain Apache tribal members live on or near the reservation. The IHS provides annual statistics of Native Americans who use any IHS services. This population data, known as the User Population, is age stratified and is recommended by the IHS as a population denominator for determining the incidence of various health conditions because it captures all persons who have used any of a wide range of health and social services over a 3-year period. The User Population includes all Native Americans who have used services at a particular facility regardless of tribal affiliation.
Case ascertainment (Fig. 1)
Step 1
We reviewed radiology records at Whiteriver IHS Hospital and SRMC to identify all CXRs taken over a 12-month period, between 1 February 2002 and 31 January 2003 (the most recent 12-month period for which full data were available at the time of study initiation), in Native Americans at least 40 years of age. A full year was studied to account for seasonality. Persons aged ⩾40 years were included because the risk of pneumococcal pneumonia increases with age, and this age group would probably be eligible for future adult pneumococcal vaccine trials.
Fig. 1. Flowchart of chest radiograph review procedure.
Step 2
A study nurse reviewed the radiology report of identified CXRs for any indication of possible pneumonia. Normal CXRs, CXRs showing stable or improving pulmonary findings and CXRs showing only non-pulmonary findings such as cardiomegaly, foreign bodies, or fractures were excluded.
Step 3
Medical records of patients who had the remaining, potentially abnormal, CXRs were retrieved and reviewed to identify eligible illness episodes (the unit of analysis). To exclude episodes of nosocomial pneumonia, illness episodes associated with the potentially abnormal CXRs were excluded if the patient was discharged from hospital within 7 days prior to the initial CXR or hospitalized for more than 72 h prior to the initial CXR. To avoid double counting events, episodes were also excluded if the patient was previously enrolled in the study within 2 months of the initial CXR. A single patient could have multiple illness episodes.
Step 4
A study nurse reviewed the remaining medical records for signs and symptoms consistent with CAP. If any of the following features were documented in the medical record for the illness episode associated with the initial CXR, then the episode was included in the study and a data collection form was completed:
• fever ⩾38°C (100·4 F),
• hypothermia <36°C (96·8 F),
• pleuritic chest pain (as indicated in the medical record),
• tachypnoea (respiratory rate ⩾20),
• change in mental status,
• new onset or increased cough,
• decreased breath sounds,
• increased sputum production,
• rales/crackles,
• dyspnoea/shortness of breath.
Data collection
A standardized data collection tool was used to collect clinical, laboratory, and radiographic information. A digital image of all CXRs associated with included episodes was created from the CXR film using a Vidar Sierra™ scanner (Vidar Systems Corporation, USA).
Clinical endpoint
Episodes were categorized as clinical CAP if there was a clinical diagnosis of pneumonia and at least one objective sign and one subjective symptom or two objective signs of CAP. Objective signs included respiratory rate >20, altered mental status, temperature >38°C or <36°C, decreased breath sounds, and rales; subjective symptoms included new or worsening dyspnoea, subjective fever, new or worsening cough, pleuritic chest pain and increased sputum production.
CXR interpretation
A standardized CXR interpretation system based on the WHO Pneumonia Vaccine Trial Investigators Group [Reference Cherian18] paediatric reading approach was developed. We made two modifications to the WHO methodology to address differences between adult and paediatric pneumonia. First, because adult pneumococcal pneumonia can present with a variety of CXR findings [Reference Lippmann20, Reference Shah21], we expanded the primary endpoint to include any consolidation, infiltrate or pleural effusion substantially obscuring lung parenchyma. The WHO definitions of these findings were used to train study radiologists. Second, because many adults have chronic lung disease, we assessed each illness episode for chronicity. The process for CXR review is outlined in Figure 1. CAP illness episodes with CXR findings showing worsening from a baseline CXR, improvement over time as evidenced by improvement in a follow-up CXR, or without comparison CXRs were categorized as acute. CAP episodes with CXR findings unchanged from baseline, or without radiographic improvement on follow-up CXRs were categorized as chronic. Acute episodes with any consolidation, infiltrate or pleural effusion obscuring any part of the lung parenchyma were categorized as meeting the primary radiographic endpoint.
Digital CXRs were read by two radiologists trained with the WHO paediatric protocol in a standardized fashion [Reference Young and Marrie17]. Discrepant readings were adjudicated by a third trained radiologist, whose interpretation was accepted as definitive. Pneumonia episodes were categorized as radiographic CAP if they met the primary radiographic endpoint and had at least one objective sign and one subjective symptom or two objective signs consistent with CAP as described above.
Data analysis
We used population data from the IHS User Population to estimate the incidence of clinical and radiographic CAP overall by age group. We examined clinical and demographic characteristics of patients with radiographic CAP and calculated kappa statistics comparing X-ray readings between the three reviewers. All analyses were conducted in Stata 9·0 (StataCorp, USA).
Ethics
The study was approved by the institutional review boards of the Phoenix Area Indian Health Service and the Johns Hopkins Bloomberg School of Public Health. White Mountain Apache tribal approval for the study was also obtained.
RESULTS
There were 193 possible episodes of pneumonia identified of which 166 met clinical screening criteria. Of these, 100 (60·2%) met criteria for clinical CAP and 60 (36·1%) met criteria for radiographic CAP. Radiographic evaluation of 73 (44%) subjects included follow-up CXR. Incidence rates for clinical and radiographic CAP are shown in Table 1. There was substantial discordance between the clinical and radiographic CAP episodes. Only 42 episodes met the criteria for both representing 42% of episodes of clinical CAP and 70% of episodes of radiographic CAP. Both absence of the primary radiographic endpoint and the presence of chronic changes were important reasons why study radiologists felt that episodes of clinical CAP did not meet criteria for radiographic CAP.
Table 1. Number of cases and incidence rates (per 1000 person-years) by age for clinical and radiographic community-acquired pneumonia (CAP) in White Mountain Apache adults from 1 February 2002 to 31 January 2003
The characteristics and outcomes of clinical and radiographic CAP episodes are shown in Table 2. Blood cultures were obtained in 52 (52%) of clinical CAP episodes and were positive in five (two Streptococcus pneumoniae isolates), while sputum was collected from 30 (30%) of which 12 had pneumococcus identified. Blood was obtained for culture from 31 (52%) radiographic CAP cases of which three were positive for bacterial pathogens (two S. pneumoniae isolates). Only 16 (27%) radiographic CAP cases had a sputum culture collected, of which eight yielded a pneumococcal isolate. Of the 153 episodes that met clinical screening criteria and had CXR available for standardized review, 69 (45%) were judged to have radiographic CAP by reviewer 1 and 58 (38%) by reviewer 2. Overall agreement between the two reviewers was 77·1%, for a kappa of 0·53. The third reviewer evaluated 34 discordant episodes and determined that 13 met the criteria for radiographic CAP. There were nine episodes that were only read by reviewers 2 and 3. Of these, the conclusion of reviewer 3 was accepted for two discordant determinations. In addition, 24 patients who did not meet clinical screening criteria had a standardized review by both reviewers 1 and 2. For these episodes, overall agreement was 78·3%, for a kappa of 0·31. The final classification of these patients was four meeting the radiographic criteria for acute pneumonia and 20 not meeting the criteria. None of these were counted as radiographic CAP cases as they did not meet the study clinical screening criteria.
Table 2. Characteristics and outcomes for patients with clinical and radiographic community acquired pneumonia (CAP) in White Mountain Apache adults from 1 February 2002 to 31 January 2003
Figures are number (%) except where otherwise specified.
* Of those with known date of vaccination, n=48 clinical CAP and n=24 radiographic CAP.
DISCUSSION
In this study, we found that the incidence of clinical and radiographic CAP in White Mountain Apache adults aged ⩾40 years was 26·7 and 16·0/1000 person-years, respectively. Persons aged ⩾65 years were at increased risk. The clinical CAP incidence rates measured in this study were substantially higher than those reported in studies of populations representative of the general US population (Table 3). We were unable to identify any comparable studies of radiographic CAP incidence from a representative US population. We did identify one comparable study of radiographic CAP incidence from another Native American population which found high rates of radiographic CAP in Navajo adults [Reference Oseasohn, Skipper and Tempest3]. However, comparisons between studies should be interpreted with caution because of differences in case definitions and surveillance methods as well as temporal variability in pneumonia incidence [Reference Foy7]. Many factors could influence CAP incidence. Whether our findings are generalizable to other Native American groups is not known.
Table 3. Incidence of community-acquired pneumonia in different populations
We may have underestimated the burden of CAP in several ways. First, we would not have identified any cases that did not undergo CXR. Because CXR is provided by the IHS at no cost to the patient, there were no financial barriers to obtaining a CXR once the patient had presented for care. However, there is no standard policy on obtaining CXR for suspected pneumonia at the Whiteriver IHS facility and clinicians may not have felt that CXR was indicated in some cases. Because this was a retrospective study, we could not influence whether CXR was done. We could not identify any efficient mechanism, using existing data, to determine the proportion of pneumonia cases that were diagnosed without CXR at these facilities. Second, cases diagnosed in a facility other than the two study hospitals would not have been identified by our surveillance system. The remote location of the White Mountain Apache reservation and the payment coverage offered by IHS reduce the likelihood that many cases were diagnosed in other facilities. Third, we did not audit the excluded cases to determine whether our method for screening CXR reports may have missed cases of pneumonia.
We did not aim to evaluate the aetiology of pneumonia in this population. As this was a retrospective study, diagnostic testing was done at the discretion of treating physicians. Blood cultures were collected in some patients, but were rarely positive. Blood culture is rarely positive in CAP [Reference Mandell12]. However, we did find a small number of subjects with a positive blood culture. Both subjects with pneumococcal bacteraemia met the criteria for both clinical and radiographic pneumonia.
Recognizing that CXRs are done for many reasons other than suspected pneumonia, we were concerned that radiographic surveillance might be an inefficient strategy for identifying possible pneumonia cases. One objective of our project was to develop an efficient system for identifying radiographic CAP that would minimize the number of medical records reviewed and CXR assessments done without sacrificing sensitivity. Even in the face of broad, highly sensitive criteria for features consistent with pneumonia, we were able to exclude 78% of CXRs by screening the onsite radiology report. This step substantially reduced the cost of the surveillance. However, we did not review any excluded CXRs to assess the sensitivity of our approach. Of the 25 patients who did not meet the clinical screening criteria for clinical CAP but who had a standardized CXR evaluation, 16% had CXR findings consistent with acute pneumonia. This observation suggests that our approach may have missed some cases of CAP.
Developing standardized, reproducible definitions of pneumonia for epidemiological studies and clinical trials is difficult. Our results showed substantial discordance between the clinical CAP and radiographic CAP case definitions. While the clinical features of clinical and radiographic CAP were similar, there were more clinical CAP cases than radiographic CAP cases, even with a radiographic endpoint that included any evidence of consolidation, infiltrate or pleural effusion. We found that study radiologists categorized some cases of clinical CAP as having no infiltrate, indicating that the clinical features of CAP can be non-specific and not associated with CXR changes [Reference Mandell12]. Study radiologists categorized some cases of clinical CAP as having stable chronic changes on CXR. It is possible that chronic lung disease can complicate both the clinical and radiographic diagnosis of CAP. Another possible explanation for discordance between clinical and radiographic CAP may be inter-observer variability in interpretation of CXRs. The two reviewers in our study only agreed on the CXR interpretation in 77% of cases. Inter-observer variability could lead to misclassification of radiographic CAP cases. Finally, it is possible that study radiologists disagreed about subtle CXR findings and the result of requiring two reviewers to concur on the presence of an abnormality selects for more prominent CXR changes. Without a gold standard definition of pneumonia, it is impossible to evaluate the sensitivity and specificity of the radiographic endpoint used in this study. The challenges of establishing standardized criteria for reading of CXRs have been well described by the WHO working group on paediatric CXRs [Reference Cherian18, Reference Baqui22]. Interpretation of adult films is complicated further by the increased prevalence of chronic lung changes. In the paediatric exercise 13/20 readers had a kappa >0·6 for agreement with the gold standard reading, compared to a kappa of 0·53 in our study. Our data suggest that additional training to reduce inter-observer variability in CXR reading may be needed before using this method more broadly. Further, additional study is needed to validate our methods of interpretation of adult CXRs in other settings and to compare it to alternative strategies.
This pilot study aimed to measure the incidence of radiographically confirmed pneumonia in a Native American population known to be at high risk for invasive pneumococcal disease, and evaluate a method for radiographic surveillance for CAP. Despite substantial discordance between reviewers of the CXRs, the incidence estimate for radiographic CAP only varied by about 15% depending on the reviewer, suggesting that this method may be useful for future studies of incidence. On the other hand, further standardization is necessary prior to employing this method in vaccine trials with CXR-confirmed pneumonia as an endpoint.
ACKNOWLEDGEMENTS
We thank the White Mountain Apache tribe for their support of this project. We also thank Lora Lavender and the staff of the Center for American Indian Health for collection of study data. Cecilia Young Kwak assisted with identifying and summarizing studies of adult pneumonia incidence. This work was supported by Sanofi-Pasteur Ltd and by the White Mountain Apache Native American Research Center for Health [National Institute of General Medicine RO1 grant U26 94 00012-01]. The opinions expressed are those of the authors and do not necessarily reflect the views of the Indian Health Service.
DECLARATION OF INTEREST
None.
