Criteria for selecting studies

Abstracts were considered eligible for full manuscript data extraction if they fulfilled the following criteria:

1. (a) the design was a cross-sectional, case–control, cohort or controlled trial

2. (b) the study population included children or adults with schizophrenia or related psychotic disorders

3. (c) second-generation antipsychotics (defined in this study as clozapine, olanzapine, risperidone and quetiapine) were being compared with first-generation antipsychotics (listed in the British National Formulary (BNF)) ¹³

4. (d) measurement of diabetes as a primary or secondary outcome was included.

Studies were excluded according to the criteria shown in Fig. 1. Studies funded by pharmaceutical companies were included, and coded as such, whether fully or partially funded.

Fig. 1 Flow chart of systematic review.

Search strategy

Using established Cochrane Collaboration search strategies and definitions for schizophrenia, antipsychotics and diabetes mellitus, we searched the following electronic libraries: MEDLINE (1966 to September 2006); PsycINFO (1967 to September 2006); EMBASE (1980 to September 2006); International Pharmaceutical Abstracts (1970 to September 2006); CINAHL Cumulative Index to Nursing and Allied Health Literature (1982 to September 2006); and Web of Knowledge (1981 to September 2006). The following search terms were used: ‘schizophrenia and related illnesses’, adapted from the Cochrane Schizophrenia Group schizophrenia search strategy; ‘antipsychotic’, including all first-generation antipsychotics in the BNF; ¹³ ‘diabetes mellitus’, amended from the Cochrane Collaboration Metabolic and Endocrine Disorder Group general diabetes search strategy (search strategy available from authors on request).

The proceedings of conferences during 2000–2005 on diabetes (American Diabetes Association, Diabetes UK, European Association for the Study of Diabetes, International Diabetes Federation) and psychiatry (American Psychiatric Association, Schizophrenia Research) were hand and electronically searched in the relevant sections.

The reference lists of included studies and reviews were searched for any additional studies. Although clozapine was first introduced in 1956 and withdrawn shortly after because of severe side-effects, there was a small risk of not detecting pre-1966 studies (when MEDLINE began), which was addressed by checking reference lists.

Corresponding authors and experts in the field were contacted for additional information on published and unpublished studies.

Data extraction

The abstracts of studies identified by electronic searches were examined by M.S. If unclear, the original article was retrieved, reviewed and discussed with co-investigators.

Data were extracted from papers selected for further review. Foreign papers were translated by doctors who were native speakers in that language.

Using a standardised data extraction sheet, we recorded and coded the following information (if available) from studies: country of origin; study design; setting; total sample size (and total number taking second-generation and first-generation antipsychotics); classification and method of assessment of schizophrenia; ascertainment of antipsychotic exposures; assessment and classification of outcome (diabetes); follow-up period; age; type of confounders adjusted; and source of funding. An attempt to retrieve missing data in the published article was made by contacting the author for correspondence on at least two occasions by email or letter.

Quality assessment

There is no consensus standardised method for the quality assessment of observational studies. We adapted the Meta-analysis of Observational Studies in Epidemiology (MOOSE) ^{Reference Stroup, Berlin, Morton, Olkin, Williamson, Rennie, Moher, Becker, Sipe and Thacker14} and Strengthening of the Reporting of Observational Studies in Epidemiology (STROBE) ¹⁵ guidelines to assess and code the quality of the studies.

A study method was considered as high-quality if the following criteria were present:

1. (a) the design was prospective, including randomised controlled trials (RCTs)

2. (b) case definition for participants with schizophrenia was based on the past three editions of the ICD or the DSM

3. (c) the study included only patients who did not have diabetes at recruitment (baseline) of the sample

4. (d) recruitment of the sample was consecutive or random

5. (e) ascertainment of antipsychotic medication exposure used an objective technique, such as screening of electronic prescription databases

6. (f) ascertainment of diabetes mellitus as an outcome was based on accepted diagnostic criteria, such as the American Diabetes Association (ADA) ¹⁶ and the World Health Organization (WHO) ¹⁷

7. (g) temporality of an association between antipsychotic and diabetes was assessed with at least 1 year of follow-up reported

8. (h) the data were controlled for the six main independent risk factors for diabetes (body mass index, family history of diabetes (first-degree relatives), ethnicity, age, physical activity and socio-economic status).

All other descriptions, or lack of any description, were coded as low-quality.

Setting and funding status of the studies were recorded but not coded for quality as it was difficult to assess with confidence how this could have introduced a bias.

We did not summarise the quality of studies with a score as this approach has been criticised as allocating equal weight to different aspects of methodology. ^{Reference Juni, Altman and Egger18}

Statistical analysis

Quantitative data from studies of high quality were pooled using random effects inverse variance weighted meta-analysis ^{Reference Woodward and Raton19} in Stata, version 9 for Windows. Where available, approximate relative risks (odds ratios or hazard ratios) and corresponding 95% confidence intervals (CIs) for diabetes, which compared second-generation antipsychotics with first-generation antipsychotics, were used from each study. Where results were given after different adjustments for confounding factors, the result for the largest confounding set was used. If only numbers with diabetes by drug type were available, these were first used to compute unadjusted odds ratios and corresponding 95% CIs for that study. Where no diabetes events occurred in any drug group, the study was excluded from the meta-analysis. Some studies gave separate results for different second-generation drugs but no overall result. In these cases, meta-analysis was used to create a pooled result, over all second-generation drugs, for that study only. This result was carried forward to the overall meta-analysis. The percentage of heterogeneity between studies not attributable to random noise was estimated using Higgins' I-squared statistic. ^{Reference Woodward and Raton19}

Strengths and limitations

The strengths of our review are that the study is based on principles derived from the Cochrane Collaboration methods, such as an a priori protocol. In addition, established Cochrane Collaboration search strategies were adapted as well as hand searching of references of relevant papers, including non-English ones. Experts in the field were contacted for missing data and further information. We also conducted separate analyses comparing individual second-generation antipsychotics with first-generation antipsychotics to take account of the unequal distribution and potentially different diabetogenic potential of individual drugs.

A possible limitation of our review is that we used a narrow search strategy which may have excluded well-designed studies that focused on related outcomes other than diabetes, such as weight gain or insulin levels. Although cardiovascular risk factors other than diabetes mellitus such as abnormal lipid metabolism, blood pressure and obesity may also have a significant effect on the physical health of patients with schizophrenia, we chose to focus on diabetes and exclude metabolic syndrome. Although the diagnosis of diabetes is relatively uniform, no such consensus exists for metabolic syndrome. ^{Reference Holt, Peveler and Byrne9} We also chose to exclude homoeostasis model assessment as an outcome as it is a research tool and its validity for predicting diabetes remains unclear. We excluded patients taking antipsychotics when the clinical indication was not clear. ^{Reference Etminan, Streiner and Rochon35} We excluded studies of mixed psychiatric populations where subgroups of patients with schizophrenia who developed diabetes could not be distinguished. ^{Reference Citrome, Jaffe, Levine, Allingham and Robinson36–Reference Spanarello, Beoni, Mina and Colotto41}

This led to some studies that included people with schizophrenia being excluded. The rationale for including only schizophrenia and related illnesses, and excluding all other psychiatric illnesses or studies which did not state diagnosis, was the importance of considering schizophrenia as an independent risk factor for diabetes. Studies that did not compare a second-generation antipsychotic with a first-generation antipsychotic, but instead to another second-generation antipsychotic or placebo were excluded. ^{Reference Brown and Estoup42–Reference Meyer45} We included all second-generation antipsychotics but there were no studies that met our eligibility criteria for the more recent drugs such as ziprasidone, amisulpride and aripiprazole, which have been suggested to be less diabetogenic in some case reports. ^{Reference Ananth, Parameswaran and Gunatilake46,Reference Mackin, Watkinson and Young47} Future studies comparing their diabetogenic profiles are awaited.

Quality of studies

We found that the average duration of studies was around 12 months with three studies having a follow up of only 3 months or less. It could be that these studies did not follow up their patients for long enough, perhaps several years, to capture the long-term risk for diabetes and this may have led to an underestimation of our findings. However, there is also some evidence that the diabetogenic effects are rapid in onset – within the first few months. ^{Reference Jin, Meyer and Jeste48} It is possible that there may be several glucose-metabolism-altering mechanisms, a short- and a long-term process, and studies with repeated measurements over several years could help to elucidate the diabetogenic patterns.

Most studies included in this review were retrospective or pharmacoepidemiological using large databases. Limitations in analysing data from such studies, mainly due to the quality of clinical data recorded, exist. Database studies do not rely on a standardised method of diagnosing diabetes mellitus. Random blood glucose measures were often used, highlighting the difficulty of obtaining fasting blood samples from this patient group. Furthermore, utilisation of established diagnostic guidelines for diabetes such as those provided by the WHO ¹⁷ or ADA ¹⁶ was inconsistent. Prescription of hypoglycaemic medication as a method of diagnosis may exclude patients receiving non-pharmacological treatment for diabetes such as those on diet alone. It may also include some people receiving metformin for other indications such as polycystic ovarian syndrome. Little evidence existed that the diagnosis of diabetes in databases was reliable or valid. The date of the first listed claim may not represent the actual date of onset of diabetes as the diagnosis may have been previously made but recorded elsewhere.

The techniques and methods used to diagnose schizophrenia cannot be inferred accurately from the databases. We observed that not all antipsychotics were included in the databases. This may be a reflection of the country where the study was undertaken and the availability of, and preference for, one type of antipsychotic over another. Many of the databases were of specific populations and mainly in the USA, which may limit their generalisability to other populations. In addition, a number of studies had only a few weeks or months of observation which may have captured those individuals who have rapid weight gain but which may be too short a time to identify later onset of diabetes.

Association between antipsychotics and diabetes

The first observation of an association between antipsychotics and diabetes occurred over 50 years ago, but renewed interest in this link has developed with second-generation antipsychotics in the past decade. Second-generation antipsychotics have been shown to reduce extrapyramidal side-effects and are beneficial to many patients. ^{Reference Geddes, Freemantle, Harrison and Bebbington49} However, the concerns that second-generation antipsychotics may also increase the risk of metabolic dysregulation must also be carefully considered.

As the majority of studies were cross-sectional or retrospective cohort in design, it is not possible to determine the direction of the association between second-generation antipsychotics and diabetes. Our review highlights that most studies were limited in quality with none of the studies fulfilling all of the quality criteria; five studies had four or fewer quality criteria (Table DS1). Any association detected in this review is likely to be significantly biased. The possibility of residual confounding also cannot be dismissed. One residual confounder not reported in any of the studies was whether an increased amount of screening occurred in those on second-generation antipsychotics compared with first-generation antipsychotics, owing to greater clinical awareness. This is important because of the high prevalence of undiagnosed diabetes in people with schizophrenia, among whom as many as 75% of cases of diabetes may be missed. ^{Reference Subramaniam, Chong and Pek50}

Our review found that second-generation antipsychotics were associated with a 30% increased risk of diabetes compared with first-generation antipsychotics in people with schizophrenia – this is probably a biased observation ^{Reference Lindenmayer, Czobor, Volavka, Citrome, Sheitman, McEvoy, Cooper, Chakos and Lieberman33,Reference Hill51} with the true association probably being smaller. This review was unable to find sufficient evidence to differentiate the risk associated with individual antipsychotics, nor did the studies we identified help shed any light on causal mechanisms. Our review emphasises that, to date, the evidence is very poor and should not be used alone as a guideline for or switching antipsychotic medications or implementing diabetes screening and management protocols for schizophrenia until further evidence comes to light.

In epidemiological terms, a relative risk of less than 2 is considered low, whereas a relative risk of greater than 3 is considered high. ^{Reference Hill51} This is an important clinical issue to consider when treating patients with antipsychotic medication. Importantly, clinicians should implement protocols for identifying physical illnesses, in particular diabetes, in people with schizophrenic illnesses, while also reviewing the rationale and dosage of prescribing antipsychotic medication both in terms of treatment of psychotic symptoms and risk of diabetes. Our review has identified the key confounders and methodological issues that need to be considered in evaluating the association between diabetes and antipsychotic medication in schizophrenia. There also remains an urgent need for continuing to identify and understand the biological pathways and increase the epidemiological evidence base for diabetes in schizophrenia. In particular, randomised controlled trials comparing first- and second-generation drugs should consider the implications of their study's power to detect differences in onset of diabetes as an adverse event, bearing in mind these events are likely to be rare.