INTRODUCTION
Leptospirosis, a spirochaetal zoonotic disease of worldwide distribution, has been recognized as an important emerging infectious disease in the last decade [Reference McBride1, Reference Sehgal2]. Humans are often exposed to leptospiral bacteria through contact with either animals or fresh water from rivers and lakes [Reference Aviat3, Reference Vanasco4]. Because of the wide variety of symptoms, leptospirosis can be easily confused with many other febrile illnesses including haemorrhagic fevers [Reference Smits5]. Timely diagnosis is essential since antibiotic therapy provides the greatest benefits when initiated early in the course of illness [Reference McBride1].
Humans react to leptospiral infection by producing specific anti-Leptospira antibodies. Seroconversion may occur as early as 5 days after the onset of disease, but may be delayed up to ⩾10 days [Reference Smits5]. During this acute phase, both IgM and IgG antibodies, can be detected in serum samples up to day 21. After 4 weeks, during the convalescent phase, a delayed immune response can arise in which both immunoglobulin classes can be detected, with a higher proportion of IgG than IgM [Reference Levett6].
The isolation of Leptospira spp. is inconvenient for clinical diagnosis because sensitivity of culture is usually low, is time-consuming and requires relatively elaborate laboratory facilities. Serological tests are therefore necessary to confirm clinically suspected cases. The microscopic agglutination test (MAT), which detects agglutinating antibodies produced against lipopolysaccharide outer antigens, is the reference ‘gold standard’ method for diagnosing human leptospirosis [Reference Aviat3] but requires the maintenance of a large number of live Leptospira strains for use as antigens and technical expertise in reading and interpreting the results [Reference Sehgal2–Reference Ooteman, Ravara Vago and Koury7].
To overcome these problems, some potentially useful tests have been developed, particularly enzyme-linked immunosorbent assays (ELISA) to detect IgM and IgG antibodies [Reference Cumberland, Everard and Levett8]. However, the efficacy of ELISA for diagnosis of leptospirosis depends on the stage of the disease, the antigen used and the class of antibodies detected [Reference Nakarin and Pradutkanchana9]. Many studies have reported the performance of ELISA applied to a variety of situations and patients. Systematic review, complemented with meta-analysis, is a recognized scientific technique of reviewing available literature using explicit methods to identify, select and critically evaluate studies which are relevant to a stated objective [Reference Faria Filho10]. We have applied this approach to evaluate the accuracy of ELISA for the detection of Leptospira-specific antibodies in humans.
MATERIALS AND METHODS
Criteria for study selection
Papers evaluating the accuracy of ELISA to detect Leptospira-specific antibodies were selected. Published studies were considered eligible if they included patients (limited to human) with proven or suspected leptospirosis and compared an ELISA test compared with MAT as the reference standard. Eligible studies were required to provide absolute numbers of diagnostic accuracy test using 2 × 2 tables. Several scientific articles reported the evaluation of more than one ELISA test against a reference standard. In those cases each comparison (e.g. using different types of ELISA test or patients in different stages of the disease) was considered separately. Therefore, the number of studies subject to analysis exceeded the number of published papers. Studies not providing relevant data on diagnostic accuracy were excluded as were reviews or duplicate reports.
Outcomes and definitions
Sensitivity, specificity, likelihood ratio for a negative and a positive result and diagnostic odds ratio (DOR) values were calculated for the ELISA tests investigated in each study, along with their 95% confidence intervals (CIs), and displayed as forest plots. The likelihood ratio for a positive or a negative result is a measure of how much the odds of the disease increase or decrease when a test is positive or negative, respectively.
Data sources
PubMed, Scopus, LILACS, and Cochrane Library databases were searched from 1980 to 2010 for articles unrestricted by language. Search terms included leptospirosis*, human and ELISA*. We also searched references from reviews and key publications on the topic. Conference abstracts were included when sufficient data were reported. Abstracts were assessed and articles that met the a priori criteria for study selection were utilized. Initial selection was based on title and abstract contents. Further selection depended upon the analysis of the original publication and the selection of those deemed to be relevant according to the selection criteria.
Data extraction
Methodological and technical data, number of patients, criteria used to select control and case groups, number of true positives, false positives, true negatives and false negatives were extracted from each study [Reference Lean11].
Statistical analysis
Data analysis was performed using Meta-Disk 1.4® software (Unit of Clinical Biostatics, the Ramon y Cajal Hospital, Madrid, Spain). Pooled sensitivity, specificity, DOR, and likelihood ratios (considered as weighted average according to size of individual studies) were calculated. These parameters can be pooled by a fixed-effects model (Mantel–Haenszel method) or by a random-effects model (DerSimonian–Laird method) to incorporate variation among studies. The fixed-effects model assumes that all studies in the meta-analysis are drawn from a common population. The random-effects model assumes that the studies were drawn from populations that differ from each other in ways that could impact on the diagnostic accuracy. In this meta-analysis we used a random-effects model, assuming that diagnostic accuracy of the test varied between studies and the various degrees of accuracy were randomly distributed among a central value [Reference Lean11–Reference Zamora13].
Each study in the meta-analysis contributed a pair of numbers: sensitivity and specificity. Since these measures tend to be correlated and vary with the thresholds used across individual studies, a summary receiver-operating characteristic (SROC) curve analysis was performed in order to explore the effect of thresholds on results. A shoulder-like curve indicates that heterogeneity between studies may be due to the threshold effect. Additionally, the Spearman rank correlation coefficient between the logit of sensitivity and logit of 1 – specificity was calculated.
Apart from the variations due to thresholds, there are several other factors that can result in variations in accuracy estimates in different test accuracy studies. Heterogeneity among studies was evaluated using the DerSimonian–Laird test (Q statistic) and inconsistency index (I 2 statistic [Reference Higgins and Thompson12]. A classification of I 2 values was used to interpret its magnitude. Values of 25%, 50% and 75% were considered as low, medium, and high heterogeneity, respectively [Reference Higgins and Thompson12]. Further reasons for heterogeneity were investigated by pre-specified subgroup (stratified) analysis. In the subgroup analysis, data was stratified according to the stage of the disease (acute or convalescent phase), class of antibodies detected by the ELISA (IgM, IgG, IgA) and type of antigen used in the development of each ELISA (crude extract or specific antigen).
Additionally, meta-regression was conducted to investigate heterogeneity. In this unweighted linear regression model, studies were the units of analysis and the DOR was the outcome variable. DOR is a unitary measure of diagnostic performance that encompasses both sensitivity and specificity and it is a suitable method to compare the overall diagnostic accuracy of different tests [Reference Zamora13]. The independent variables (stage of the disease, type of immunoglobulin, type of antigen) were the covariates that might be associated with the variability in DORs. The result of the meta-regression model was reported as relative diagnostic odds ratio (RDOR) [Reference Flores14].
RESULTS
The literature search yielded 96 scientific papers on ELISA used to detect Leptospira antibodies in humans; of these 49 failed to meet one or more of the inclusion criteria (Fig. 1). A further 12 articles were excluded because they either had other objectives, were conducted in animals, showed duplicate data, or lacked sufficient statistical information to conduct a meta-analysis. Thirty-five articles describing 88 studies involving 21 494 patients were therefore available for further analysis. Two studies were conducted before 1990, eight between 1991 and 2000 and the remaining 25 after 2001. IgM ELISAs were evaluated in 21 instances whereas only three studies evaluated IgG ELISAs. Nine papers reported evaluations of IgM and IgG ELISAs and one IgG, IgM and IgA ELISA; one study only used an IgA ELISA.
Fig. 1 Study selection flow chart.
In 17 studies the stage of disease was unknown or not specified. Three and two studies were reported on patients who had leptospirosis in acute and convalescent phases, respectively and 13 were conducted in patients with both stages of the disease (Table 1). Thirty studies utilized ELISA using whole-cell Leptospira-based antigens and five using recombinant/synthetic antigens.
Table 1. Description of studies included in the meta-analysis
Most of the studies were conducted in Brazil (n = 11), and others were conducted in Thailand (n = 3), India (n = 3), Argentina (n = 2), USA (n = 2), and Barbados (n = 2). Two studies used samples from different regions (Hawaii, Indonesia, Seychelles, The Netherlands). Single studies were conducted in each of the following countries: Venezuela, Seychelles, The Netherlands, Singapore, Laos, UK, Peru, Italy, France, and Iran.
Accuracy of assays
The analysis of the 88 assays yielded a pooled sensitivity of 0·779 (95% CI 0·770–0·789) (Fig. 2) and specificity of 0·913 (95% CI 0·908–0·917) (Fig. 3) for the end-point prediction. Positive and negative likelihood ratios were 12·9 (95% CI 9·935–16·750) and 0·206 (95% CI 0·175–0·243), respectively. The DOR was 75·057 (95% CI 53·181–105·930) indicating high accuracy. SROC analysis showed that the area under the curve (AUC) was 0·964, Q* value was 0·910 (SE(Q*) = 0·0117). Asymmetric SROC analysis yielded no difference in the result (Fig. 4). The significant heterogeneity in sensitivity and specificity estimates precluded the determination of clinically useful summary measures.
Fig. 2 [color online]. Forest plot of sensitivity estimate for ELISA diagnosis of human leptospirosis. •, Point estimates of sensitivity from each study (proportional to size of the study); ——, 95% confidence intervals;◆, pooled sensitivity estimated.
Fig. 3 [color online]. Forest plot of specificity estimate for ELISA diagnosis of human leptospirosis. •, Point estimates of specificity from each study (proportional to size of the study); ——, 95% confidence intervals;◆, pooled specificity estimated.
Fig. 4 [color online]. Summary receiver-operating characteristic (SROC) plot for ELISA diagnosis of human leptospirosis. •, Each study in the meta-analysis, size proportional to size of study; ——, regression line that summarizes the overall diagnostic accuracy.
Heterogeneity and stratified analyses
One of the primary causes of heterogeneity in test accuracy studies is the threshold effect. The latter arises when differences in sensitivities and specificities are due to the selection of different cut-off values to define a positive (or negative) test result [Reference Zamora13]. The Spearman correlation coefficient between the logit of sensitivity and log of 1 – specificity was not significant (P = 0·062). Studies clearly showed a high degree of variability in sensitivity (P < 0·001, I 2 = 93·72%) and specificity (P < 0·001, I 2 = 95·02%) estimates. This heterogeneity may result from differences in test methods (type of immunoglobulin identified by the ELISA test, and type of antigen used in the development of each test) or population characteristics (patients sampled in the acute or convalescent stage). In order to identify factors associated with heterogeneity, stratified (subgroup) analyses were performed (Table 2).
Table 2. Summary of measures for sub-analysis comparison
CI, Confidence interval.
Three subgroup analyses restricted to the stage of the disease revealed 37 studies where disease stage was not identified or included patients in different stages. The sensitivity and specificity values in this subgroup were 0·802 (95% CI 0·788–0·816) and 0·869 (95% CI 0·860–0·878), respectively. When patients in the acute stage were considered (n = 26), the pooled sensitivity was 0·683 (95% CI 0·663–0·701) and the pooled specificity 0·946 (95% CI 0·939–0·952). However, when the ELISA was applied in convalescent patients (n = 25) sensitivity was higher (0·863, 95% CI 0·847–0·879) than in the acute phase, while specificity remained at similar levels (0·934, 95% CI 0·925–0·941). However, all the stratified analyses showed significant heterogeneity (Table 2).
Four subgroup analyses restricted to the type of immunoglobulin identified by the ELISA test (IgM, IgG, IgA, or IgM + IgG), were conducted. Two assays using IgM and IgG in the same ELISA showed a pooled sensitivity and specificity of 0·876 (95% CI 0·860–0·995) and 0·555 (95% CI 0·475–0·632), respectively. The pooled sensitivity (0·804, 95% CI 0·792–0·815) and specificity (0·944, 95% CI 0·939–0·949) were higher in the assays which used an IgM rather than IgG ELISA (sensitivity = 0·736 and specificity = 0·848). Another subgroup analysis restricted to only two assays based on IgA ELISA showed the lowest sensitivity (0·685, 95% CI 0·578–0·780) and the highest specificity (0·990, 95% CI 0·948–1·000). However, the stratified analysis did not reduce the heterogeneity in the studies (Table 2).
Taking into account the type of antigen used in the development of each ELISA test, the subgroup analysis indicated no effect on the sensitivity when crude extract antigen (0·794, 95% CI 0·783–0·805) or specific antigens (0·744, 95% CI 0·725–0·762) were used. However, those assays which used crude extract antigen had increased specificity (0·920, 95% CI 0·914–0·925) over ELISA based on specific antigens (0·890, 95% CI 0·880–0·901). Nevertheless, the type of antigen used had no significant influence on heterogeneity (P < 0·001) (Table 2).
None of the stratified analyses results fully explained the significant heterogeneity across studies in this review. The statistical tests for heterogeneity were significant even within the different strata (Table 2). Therefore, a meta-regression analysis was performed to evaluate multiple factors in the same analysis. The outcome of the regression analysis was the RDOR (Table 3). Studies that included patients in the convalescent stage of the disease showed a RDOR significantly higher (1·84 times) than those that included patients in the acute stage or where disease stage was not identified. Studies that utilized an IgM ELISA produced RDOR values significantly higher (7·14 times) than assays that detected IgG. The type of antigen used in the ELISA did not produce a significant RDOR, indicating that the use of crude extract or specific antigen did not substantially affect diagnostic accuracy.
Table 3. Meta-regression analysis to determine sources of heterogeneity
RDOR, Relative diagnostic odds ratio; CI, confidence interval.
Intercept = constant in the model.
S, Indicator of threshold (logit true positive rate + logit false positive rate).
DISCUSSION
This meta-analysis was conducted taking into account the standard protocol for systematic reviews, considering studies published in different languages. Two reviewers independently performed the article selection and data extraction. Several methodologies were applied, including SROC analyses, methods for exploring heterogeneity and meta-regression.
Our meta-analysis, based on 88 published ELISA evaluations, showed a SROC curve with an AUC of 91%, indicating a high efficiency for the detection of leptospirosis in human patients. However, the high heterogeneity in the overall sensitivity and specificity estimates may hamper their usefulness as summary measures for clinical detection. In an attempt to explain the observed variability, we performed a stratified analysis with three major sources of variability: (1) phase of the disease (not identified, acute and convalescent), (2) type of immunoglobulin detected by the ELISA (IgM, IgG, IgA, or IgM + IgG) and (3) type of antigen used in the development of each assay (whole cell or recombinant/synthetic). Heterogeneity was evident in the results and could not be completely explained by the different sources of variability analysed.
The variability and quality in study designs can be a source of heterogeneity of results. However, no categorization taking into account the quality of studies was included in our analysis due to the lack of sufficient elements for such an assessment. Several other variables that could generate heterogeneity among the diagnostic methods include demographics (age, sex, occupation), clinical severity and morbidity, and regional characteristics such as climate and flooding. These sources could not be quantified or measured in this study owing to the lack of information available and merit further investigation.
Due to the heterogeneity in sensitivity and specificity, it is difficult to determine clinically useful estimates of accuracy. Nevertheless, meta-regression analysis allowed us to highlight some variables that apparently affect the estimates of efficiency of ELISAs in diagnosing human leptospirosis. This technique showed a significantly higher efficiency when applied to patients in convalescent rather than acute or unidentified phases. The latter is expected for any serological test because of the time lapse for the appearance of serum antibodies after an infection and increased sensitivity does not necessarily correlate with time [Reference Fainboim and Geffner43].
Another factor associated with the effectiveness of an ELISA is the type of antibody detected. According to their DOR, assays which identified IgM were seven times more efficient than those that detected IgG. This marked difference indicates that, in most studies, sensitivity and specificity of IgM ELISA were significantly greater than for IgG assays. Anti-Leptospira IgM antibodies appear earlier than IgG and remain detectable for months or years at low titres. On the other hand, IgG levels may vary from non-detectable, or positive for short time periods or several years [44].
However, the higher specificity of IgM over IgG ELISA is difficult to explain because IgM in general is characterized by a lower affinity than IgG for antigens [44]. In this system patient IgM and IgG responses compete for binding to antigen and the higher affinity IgG molecules are more readily captured. Non-specific IgM is unable to bind and is therefore not detected. In this case, non-specific IgG is reflected as a false-positive test result for IgG which is not shown for IgM. Thus, what constitutes a classical advantage for IgG detection could become a disadvantage that indirectly favours IgM detection. A possible explanation for this is that patients with febrile diseases other than leptospirosis may have non-specific IgG antibodies due to a false-positive test result.
Finally, the type of antigen (whole cell or recombinant/synthetic) used in the ELISA test did not produce a significant change in the RDOR. These results are consistent with recent studies suggesting that recombinant and synthetic antigens would fail to overcome the diagnostic efficiency of crude extract antigen, which is a whole-cell antigen of Leptospira [Reference Vanasco45]. Faced with a suspected case of leptospirosis, an IgM ELISA will show good efficacy and be the best choice of test; the type of antigen would not be of great importance nor the stage of disease. However, if the patient is in the acute phase and both IgG and IgM are negative, leptospirosis can not be ruled out due to the possibility of false-negative results in the early stages of the disease. In this case it is advisable to request a second blood sample or apply a direct method such as real-time PCR for leptospiral DNA [Reference Vanasco46].
In conclusion the meta-analysis identified the type of detected immunoglobulin and the phase of the disease in the patient to be two important sources of variability for the efficiency of ELISA for the diagnosis of human leptospirosis. Our results suggest that an IgM ELISA would be the best choice of method for early detection of cases during the acute phase of infection and serve as a good screening tool at all stages of the disease. Additional studies should be performed to analyse the effect of other potential sources of variability, especially those related to regional aspects.
ACKNOWLEDGMENTS
Marcelo L. Signorini is a Research Career Member from the Consejo Nacional de Investigaciones Científicas y Técnicas (CONICET, Argentina).
DECLARATION OF INTEREST
None.
