Background
Unplanned readmission is often seen as an undesirable outcome for both the patient and the healthcare system. For the patient, being readmitted soon after discharge is disruptive to their recovery and, in the long run, is detrimental to the course of illness with poorer outcomes and may even engender dependency on psychiatric services.Reference Evans, Harris, Newman and Beck1,Reference Frick, Frick, Langguth, Landgrebe, Hübner-Liebermann and Hajak2 Frequent readmissions put a strain on the family, the patient's social support network and lead to social stigma.Reference Kalseth, Lassemo, Wahlbeck, Haaramo and Magnussen3 The staff may also feel demotivated over their confidence in the management of the patient.Reference Lien4 Rapid readmission, especially for patients readmitted to hospital within a month of discharge, reflects poorly on the quality of care provided by the psychiatric unit during the in-patient stay, with a lack of coordination between the discharging unit and the community care providers and may be rather costly.Reference Callaly, Trauer, Hyland, Coombs and Berk5–Reference Hamilton, Rhoades, Galvez, Allen, Green and Aller7 High rates of unplanned readmissions may also indicate premature discharge from the psychiatric units as hospitals in certain countries have been encouraged to shorten the patient length of stay and hospitals with high readmission rates may be penalised financially.Reference Donisi, Tedeschi, Wahlbeck, Haaramo and Amaddeo8
The rate of psychiatric patients readmitted within 28 days of discharge is one of the established indicators for quality of care collected by many countries, including the USA, UK and Australia, which have set up their own benchmarking tools for comparisons across different hospitals.9–11 Data from the Organisation for Economic Co-operation and Development (OECD) in 2011 showed the readmission rates within 28 days of discharge for patients with schizophrenia and bipolar affective disorder, two diagnoses with higher readmission rates, were 13 and 11%, respectively.12 The actual figures for individual countries differ according to different research, which has reported rates up to 24.6% for patients with psychosis in the USA.Reference Lorine, Goenjian, Kim, Steinberg, Schmidt and Goenjian6 However, given constraints in resources, any intervention to decrease readmission rates for these illnesses should be targeted at those with a high risk.
Aims
This study aims to create a risk model that can predict the risk of unplanned readmissions via accident and emergency departments within 28 days of discharge from acute psychiatric units for patients with psychotic spectrum disorders, so that those with a particularly high risk and need may be identified. There has only been one risk index published to predict 30-day readmission after discharge from acute psychiatric units developed from a cohort of patients from Ontario, Canada, encompassing all psychiatric diagnosis.Reference Vigod, Kurdyak, Seitz, Herrmann, Fung and Lin13 The model was then applied to a different group of patients in south-eastern USA with good predictive ability.Reference Roque, Findlay, Okoli and El-Mallakh14 Risk models are already being deployed in some countries. Scotland has deployed a risk model to calculate the risk of readmissions for all diagnoses, including psychiatric hospital readmission.15 However, healthcare needs and specific interventions differ for patients with different diagnoses. To our knowledge, there are no risk models published globally that target patients with psychotic spectrum disorders in particular.
Method
Study design and setting
This is a retrospective cohort study conducted across all eight public psychiatric units across Hong Kong that provide an in-patient treatment facility. The vast majority of the in-patient psychiatric care is provided via the public sector in Hong Kong, operated by the hospital authority. The hospital authority also operates general out-patient clinics and specialist out-patient clinics, including 11 psychiatric out-patient clinics, which share the same electronic health record platform with in-patient services. Similar to the UK, the territory of Hong Kong has been split into seven catchment areas, each with its own in-patient and out-patient services. With a population of about 7.5 million people, Hong Kong operates a heavily government-subsided healthcare system where all residents in Hong Kong are guaranteed healthcare, regardless of individual's financial capabilities. Following the global trend of deinstitutionalisation of mental healthcare, there are approximately 50 psychiatric beds per 100 000 population in Hong Kong, more than the proportion in the UK or USA but lower than the OECD average of 69 psychiatric beds per 100 000 population.16,17
Data sources
The majority of the data sources for this study came from the Clinical Data Analysis and Reporting System (CDARS). CDARS compiles data from multiple online systems, including the clinical management system, the in-patient management system and the corporate drug dispensing system, dating all the way back to 1994. As all of the public healthcare facilities in Hong Kong are operated by the same healthcare provider, the system is able to pool data across every public hospital in the city. Aside from the patients’ medical records, the system is also able to retrieve administrative records relating to every single patient. Currently, CDARS is unable to retrieve and analyse free text entered into the various information systems, such as the discharge summaries, but can retrieve other categorical information. Any duplications in the data retrieval process from CDARS were cross-checked or supplemented with information from the actual free-text discharge summaries.
CDARS is also able to retrieve information from the Psychiatric Clinical Information System (PsyCIS). PsyCIS is the system that incorporates every patient's mental health history and treatment records. The system contains detailed admission and discharge records for every in-patient episode, categorical information on the patient's pertinent past history (such as any history of violence or suicide), details of the current admission and services received upon discharge. Every treating psychiatrist is also required to fill in the patient's admission and discharge Health of the Nation Outcome Scales (HoNOS).Reference Wing, Beevor, Curtis, Park, Hadden and Burns18
Cohort
This study reviewed data for patients aged 18 or above and under 65 years old who were discharged from territory-wide psychiatric units over a 5-year period between 1 January 2013 and 31 December 2017. Those who have undergone an episode of unplanned readmission within 28 days of discharge were flagged by CDARS. Only patients with a primary diagnosis of psychotic spectrum disorders were included in the study, as research has suggested that the presence of psychotic experiences, even for patients whose primary diagnoses are affective disorders, is associated with worsened outcomes.Reference Van Os and Reininghaus19 Therefore, under the current ICD-10 diagnostic criteria,20 patients with a primary diagnosis shown in Table 1 were included.
Table 1 Patient diagnosis included in the study
Only patients readmitted via the accident and emergency department were included, rather than through the out-patient setting, ward follow-up's or clinically admitted for procedures, where readmissions were regarded as planned.10 Only patients who stayed at the hospital for more than 3 days were included in the study, as the characteristics of patients who stayed for a short time differed substantially from patients with a longer length of stay and such short admissions often do not allow for enough time for the patient to be meaningfully assessed and treated.Reference Vigod, Kurdyak, Seitz, Herrmann, Fung and Lin13 Patients with moderate or severe degrees of intellectual disability were excluded from the study as their admission may be because of behavioural problems, rather than a relapse of any comorbid psychotic spectrum disorders. For patients transferred to general hospital for acute medical conditions during their stay in a psychiatric unit and subsequently transferred back to a psychiatric unit this was counted as one episode.
Predictor variables
The potential predictor variables selected for analysis were based on the grounds of clinical relevance, the results from the literature review and the availability of clinical data. The predictors could be broadly split into four groups of factors. The first group included data on the sociodemographic background of each patient, including age, gender, education, whether the patient's hospital admission fees were waived because they were on social welfare and where they were living after discharge. Those who were on social welfare would be exempted from any hospital charges under the hospital authority system. However, patients who have been involuntarily admitted into psychiatric units would also have their hospital admission fees waived, regardless of whether or not they receive social security. As the hospital authority system is unable to directly show whether the patient is on social security, the actual social welfare status of the patient was not used as a predictor in this study. The second group of predictors was related to the past medical and psychiatric history of the patients.
Physical comorbidities were quantified using the Charlson score, a measure of physical comorbidity that has been widely used in many different publications.Reference Romano, Roos and Jollis21–Reference Barker, Gruneir, Fung, Herrmann, Kurdyak and Lin24 The third group of variables was related directly to the current discharge episode, which included the legal status of the patient upon discharge, the ‘special care system (SCS)’ status of the patient and whether a conditional discharge was signed for the patient. SCS is a system that provides a framework to manage the risks and needs of patients receiving specialist psychiatric services, with input from doctors, social workers and staff from non-governmental organisations. Those who have been labelled special care or intensive care under the SCS system, the two higher levels out of a three-level system, receive more community support from other allied health practitioners, akin to the assertive outreach team in the UK, with intensive care reserved for patients with a very high degree of clinical complexity and a risk of committing serious violence.25 Similarly, the conditional discharge system in Hong Kong is analogous to the community treatment order in the UK, typically reserved for patients with a propensity for violence. These patients may be recalled into the specific psychiatric units should they breach certain conditions of their discharge. The last group of variables pertained to the discharge HoNOS score of the patients.
Statistical analysis
We first examined the normality of data of the variables in our sample using Shapiro-Wilk Test. χ²-tests (for categorical data), independent t-tests (for parametric data) and Mann-Whitney U-tests (for non-parametric data) were used to detect any significant difference in each predictor between those readmitted and those who were not readmitted. Variables with more than 25% missing data were excluded from the analysis. Multivariate logistic regression models were then applied to construct a risk prediction model. To build and internally validate the predictive model, the analysis used a split-sample method, whereby the entire cohort was randomly split into two, half of which was used to derive the model whereas the other half was used to validate the model. This study used patient episodes, rather than individual patients, as the main analysis, a method commonly used by many other readmission models as well.Reference Zhou, Della, Roberts, Goh and Dhaliwal26 A subsidiary analysis was also performed using patient-level data, taking only the first discharge episode within the 5-year period of a particular patient if there were more than one episode, so as to mitigate the effects of frequent admitters.
A series of four models was created by sequentially adding each group of variables mentioned above. This method was similar to the model derivation proposed by Vigod et al,Reference Vigod, Kurdyak, Seitz, Herrmann, Fung and Lin13 which aimed to prioritise the variables that could be easily collected in the clinical setting requiring the least manual input while still maximising the predictive capacity of the model. The predictive model was constructed using backwards logistic regression, with the probability set at 0.05 for removal of factors. The choice of the final model was decided depending on the clinical and statistical significance of the predictors, the goodness-of-fit using the Hosmer–Lemeshow goodness-of-fit test and the model discriminative ability by the area under the receiver operating characteristic curve (AUC), also known as the c-statistic. A defined cut-off point, determined by comparing against the actual incidence of unplanned readmission, for dichotomising high risk and low risk was used to determine the sensitivity, specificity, positive predictive value (PPV) and negative predictive value (NPV) of the model. The probability for readmission for patients in the validation set was then calculated using the model from the derivation set. The final probability was then compared against the actual result of whether the patient has been readmitted or not, with another AUC computed for validation of the predictive model. All statistical analyses were carried out using Statistical Package for the Social Sciences (SPSS) software version 22.
The study has received approval from the research and ethics committees of all the study sites of this study in Hong Kong, namely New Territories West Cluster Clinical and Research Ethics Committee, Joint Chinese University of Hong Kong New Territories East Cluster Clinical Research Ethics Committee, Kowloon West Cluster Research Ethics Committee, Research Ethics Committee (Kowloon Central/Kowloon East), Institutional Review Board of the University of Hong Kong/Hospital Authority Hong Kong West Cluster and Hong Kong East Cluster Research Ethics Committee. As there was no actual patient contact in this study and only the case notes were perused, informed consent from every patient has been specifically waived. The data has been recorded in a manner that ensures that each individual could not be identified, using a non-recognisable code for each patient.
Results
Participant characteristics
Our sample comprised 30 707 discharge episodes, relating to 18 514 individual patients. Out of the 30 707 discharge episodes, the incidence of 28-day unplanned readmission via the accident and emergency department was 7.09%, with 1496 individual patients experiencing one or more readmission episodes. With more than 25% missing data, duration of illness was excluded in the generation of the regression models.
Table 2 describes the sociodemographic characteristics of the patients, Table 3 the past medical and psychiatric history and current discharge episode and Table 4 the HoNOS scores for the patient who were readmitted and those who were not. Taking both patient groups together, there were more women than men admitted to hospital. Most patients studied up to secondary school level and lived with their family or relatives after discharge (Table 2). Patients were diagnosed with schizophrenia, schizotypal or delusional disorders in 90.7% of the episodes, and substance misuse was the most common comorbid psychiatric illness (Table 3). Approximately 36% of patients were put on depot medications. HoNOS scores were recategorised into 0 (no problems experienced) or 1–4 (minor to severe degrees of problems experienced) as most patients were ranked 0 upon discharge (Table 4). Most factors showed significant difference between the readmitted group and the non-readmitted group (with P < 0.05).
Table 2 Socioeconomic characteristics associated with those readmitted and not readmitted
* P < 0.05.
a. Readmitted: n = 2048; not readmitted: n = 26 636.
b. Readmitted: n = 1599; not readmitted: n = 221 902.
Table 3 Past medical and psychiatric history and current discharge information characteristics associated with those readmitted and not readmitted
* P < 0.05.
a. Not readmitted: n = 28 459.
Table 4 Health of the Nation Outcome Scales (HoNOS) scores associated with those readmitted and not readmitted
* P < 0.05.
Derivation of the risk prediction model and measuring the model's accuracy
Owing to missing data, the final number of episodes used to derive the logistic regression was 10 219. The results of each regression model are presented in Table 5. The P-values for the Hosmer–Lemeshaw tests of all four models were above 0.05, meaning that the fitted models were satisfactory and the observed and predicted values of the dependent variables were different. The addition of more groups of variables resulted in higher c-statistics, indicating a logistic regression model with more predictive variables resulted in better model discriminative power. As a result, model 4 was chosen as the final model for the prediction of unplanned readmissions. The model had a c-statistic of 0.705, indicating a moderate discriminative power. Taking a cut-off point of 8% as the risk of readmission, similar to the actual readmission risk of 7.09% during the study period, the model achieved a sensitivity of 47.2%, specificity of 79.3%, PPV of 14.8% and a NPV of 95.2%. When the model was applied to the validation set, which consisted of the other half of the total episodes (n = 10 643), the c-statistic was 0.684, again indicating a moderate discriminative power.
Table 5 Results of the Hosmer–Lemeshow test and discriminative power of the various models
AUC, area under the receiver operating characteristic curve; HoNOS, Health of the Nation Outcome Scales.
Table 6 shows the predictive variables that were statistically significant in model 4, the final prediction model. All predictors achieved statistical significance with a P-value of less than 0.05 apart from certain variables in the abode after discharge. Patients with more admissions in the past or who had comorbid substance use disorders were more likely to be readmitted. Those with a history of violence or who exhibited signs of agitation or aggressiveness upon discharge, as signified by a score of 1 or more in the first item of HoNOS, were also more likely to be readmitted within 28 days. A prescription of clozapine, older age or the imposition of conditional discharge status prior to discharge conferred a protective effect. Those who were discharged to live with their family or relatives were less likely to be readmitted than those who lived alone.
Table 6 Predictors of unplanned readmission within 28 days of discharge
* P < 0.05.
Subsidiary analysis
In the subsidiary analysis using patient-level data, there were 18 514 patients included in the analysis. Half of these patients was used to derive the prediction model. As with the episode-level data, there were significant amounts of missing data. A total of 5548 and 6936 patients were included in the final derivation model and validation model. Higher number of previous admissions, comorbid substance misuse and history of violence were once again found to be the risk factors for unplanned readmission along with no fixed abode after discharge, whereas older age and living with family or relatives provided a protective effect. Even though conditional discharge was included in the model, the variable did not achieve statistical significance. The model had a P-value of 0.836 on the Hosmer–Lemeshaw test, indicating a model with satisfactory fit. The c-statistic of the derivation and validation model was 0.658 and 0.661, respectively, both showing moderate discriminative power.
Discussion
Main findings
This study has derived and internally validated a risk model using a split-sample approach to predict unplanned readmissions within 28 days of discharge from psychiatric units via the accident and emergency department for patients with psychotic spectrum disorders; this is the first model published in the literature to predict the risk of readmission for this particular group of patients. The c-statistics of the derivation (0.705) and validation (0.684) set both indicated that the model was of moderate discriminative power, on a par with other prediction models for readmissions, including those for other medical conditions and higher than the model used to predict readmissions for all psychiatric diagnoses, where the c-statistic was 0.631 and 0.630 for the derivation and validation sets.Reference Vigod, Kurdyak, Seitz, Herrmann, Fung and Lin13,Reference Zhou, Della, Roberts, Goh and Dhaliwal26,Reference Kansagara, Englander, Salanitro, Kagen, Theobald and Freeman27
Like many other predictive models, the model derived in this current study has a moderate sensitivity (47.2%), low PPV (14.8%) but a relatively high specificity (79.3%) and NPV (95.2%) using a cut-off point of 8% as the risk of readmission. The moderate sensitivity means that only half of the patients subsequently readmitted were identified by the model, although this may be improved by adjusting the cut-off point. The relatively low incidence of unplanned readmissions imposed a ceiling on the PPV and many of the patients with a positive test were eventually not readmitted. This combination of moderate sensitivity, low PPV and high NPV is similar to prediction models for suicide and violence, demonstrating that risk predictions are generally difficult in mental health given the multifactorial associations.Reference Fazel, Singh, Doll and Grann28,Reference Belsher, Smolenski, Pruitt, Bush, Beech and Workman29
Given the findings from this study, there are certain interventions that psychiatrists may deliver to their patients, particularly those with a high risk, to decrease the rate of unplanned readmissions, such as psychoeducation, medication education and reconciliation and coping skills training that have been proven to be effective.Reference Vigod, Kurdyak, Dennis, Leszcz, Taylor and Blumberger30–Reference Schmidt-Kraepelin, Janssen and Gaebel32 Psychiatrists should also place more emphasis on the treatment of comorbid substance use disorders and find ways to manage any residual aggression or overactivity upon discharge, two risk factors found to be significant in this study. Other effective measures noted in the literature, such as post-discharge telephone follow-up and efforts to ensure timely follow-up, may be provided by other allied health professionals or mental health hotlines.Reference Vigod, Kurdyak, Dennis, Leszcz, Taylor and Blumberger30 These interventions have a low cost and low likelihood of adverse events and can be delivered to many of those predicted with a high risk of readmission. Even if some of patients were falsely predicted to be readmitted, because of the low PPV, it is unlikely that they would be harmed with these practices and those who tested negative would not receive unnecessary interventions given the high NPV.Reference Carter, Milner, McGill, Pirkis, Kapur and Spittal33
Interpretation of our findings and comparison with findings from other studies
Apart from the large sample size used to derive the prediction model, another strength of this study is the inclusion of both pre-admission and treatment information in the prediction retrieved from an objective clinical population-level database. If unplanned readmissions are used as an indicator for the quality of care provided, then clinicians should be able to alter their practice in a meaningful way to decrease the occurrence of the event.Reference Durbin, Lin, Layne and Teed34 Models that only incorporate historical data and ignore information related to the current episode may not be able to suggest any clinically and statistically significant interventions that can decrease unplanned readmissions. The research published by Vigod et al on readmissions for all psychiatric diagnoses incorporated certain parameters related to the index hospital admission, such as the diagnosis, length of stay, threat to self or others, planned discharge and self-care ability, but omitted other treatment factors and left out the results from the rating scales on the patient's symptomatology.Reference Vigod, Kurdyak, Seitz, Herrmann, Fung and Lin13 The predictor variables are also readily retrievable and the risk model may therefore be applied or validated in other countries.
The results from this study show that prescribing clozapine and establishing a conditional discharge requirement on the patient may be protective and decrease the risk of unplanned readmissions. A large trial comparing the effectiveness of antipsychotic treatments for patients with schizophrenia has already proven that clozapine is associated with the lowest rates of treatment failure, as defined by many parameters including psychiatric hospital readmission.Reference Tiihonen, Mittendorfer-Rutz, Majak, Mehtälä, Hoti and Jedenius35 Other research conducted in the UK found that patients with schizophrenia or schizoaffective disorder who have been newly prescribed with clozapine have a decreased risk of readmission, even if these patients had more severe psychopathology and poorer functional status.Reference Kesserwani, Kadra, Downs, Shetty, MacCabe and Taylor36 A systematic review on the impact of clozapine on hospital use also found that clozapine reduced the proportion of people admitted to hospital compared with controls.Reference Land, Siskind, McArdle, Kisely, Winckel and Hollingworth37 The authors hypothesised that the reason for this protective effect may be because of the need for ongoing regular haematological monitoring, where deteriorations in mental state may be detected earlier. Clozapine is also known to reduce aggressive behaviour in patients, independent of its antipsychotic effect.Reference Meyer and Stahl38
This study has also shown the protective effect of conditional discharge orders. It is possible that patients with this order may have received more robust assessments before discharge and be more adherent with their treatment and follow-up attendance, thereby reducing the risk of readmission. However, a Cochrane review on compulsory community and involuntary out-patient treatment for people with severe mental disorders found that there was no clear statistical difference in readmissions to hospital by 12 months.Reference Kisely, Campbell and O'Reilly39
The protective effect of older age or living with family or relatives in reducing readmissions has been described in some literature, although other studies have often showed mixed results.Reference Donisi, Tedeschi, Wahlbeck, Haaramo and Amaddeo8,Reference Sfetcu, Musat, Haaramo, Ciutan, Scintee and Vladescu40 Similarly, the increased risk of readmission associated with a history of violence, residual symptoms of overactivity or agitation as measured by HoNOS item 1 and comorbid substance misuse has been described in some literature, although these findings were often not found in other studies.Reference Donisi, Tedeschi, Wahlbeck, Haaramo and Amaddeo8,Reference Tulloch, David and Thornicroft41,Reference Burke, Donzé and Schnipper42 The only consistent variable that has a proven predictive effect is a higher number of previous admissions, sometimes over a pre-defined period in the past, for example 3 years.Reference Zhou, Della, Roberts, Goh and Dhaliwal26,Reference Donisi, Tedeschi, Salazzari and Amaddeo43,Reference Moss, Li, Tobin, Weinstein, Harimoto and Lanctôt44 Interestingly, one piece of research found that interventions such as symptom education and service continuity, previously noted to be effective to prevent readmissions, may become statistically insignificant for those with four or fewer previous admissions, suggesting that the benefits may be dependent on the pattern of use of healthcare services.Reference Prince31
The length of stay of the patient was not found to be significant in this study, even though some studies have showed that a shorter length of stay during the index hospital admission led to a higher risk of readmission.Reference Kalseth, Lassemo, Wahlbeck, Haaramo and Magnussen3,Reference Tulloch, David and Thornicroft41,Reference Hamilton, Passos, De Azevedo Cardoso, Jansen, Allen and Begley45 This has become an important finding as many countries have pressed for a shorter length of stay and some psychiatrists may discharge a patient prematurely before recovery in an attempt to hasten the patient's reintegration into society in the community.Reference Rylander, Colon-Sanchez, Keniston, Hamalian, Lozano and Nussbaum46 This may be because of the relatively long length of stay of patients in Hong Kong compared with other countries, with less pressure for early discharge, and data being partly skewed by the discharge episodes of certain patients who have lived in psychiatric institutions for years.Reference Tulloch, David and Thornicroft41
Depot medications were also not found to be significant in preventing readmission, contrary to the findings of nationwide cohort studies or publications solely comparing the risk of readmission to hospital for patients taking oral antipsychotics versus depot medications.Reference Tiihonen, Mittendorfer-Rutz, Majak, Mehtälä, Hoti and Jedenius35,Reference MacEwan, Kamat, Duffy, Seabury, Chou and Legacy47 However, other papers have found that depot medications had no association with the risk of readmission.Reference Roque, Findlay, Okoli and El-Mallakh14,Reference Hung, Chan and Pan48 Depot medications are sometimes prescribed for patients with a high risk of poor drug adherence, which some have suggested to be the cause for relapse. However, some pieces of research have failed to find an association, suggesting that medication non-adherence is more likely to be associated with delayed readmissions to psychiatric units, rather than rapid readmissions such as the time frame of 28 days used in this study.Reference Lorine, Goenjian, Kim, Steinberg, Schmidt and Goenjian6,Reference Hamilton, Passos, De Azevedo Cardoso, Jansen, Allen and Begley45,Reference Craig, Fennig, Tanenberg-Karant and Bromet49
Although there is a statistically significant result on the χ²-test for the effect of gender and readmission, logistic regression has failed to demonstrate gender as a significant predictor of readmission. The female predominance of discharges is similar to the overall statistics in Hong Kong across all diagnoses, possibly because women are more likely to seek help for their medical problems with better insight and exhibit more bizarre behaviour and affective symptoms upon presentation, thus leading to admission to hospital.Reference Barajas, Ochoa, Obiols and Lalucat-Jo50–Reference Chang, Tang, Hui, Chiu, Lam and Wong52
Comparison of patient-level and episode-level data results
Our subsidiary analysis using patient-level data rather than episode-level data revealed that the predictors of unplanned readmission included comorbid substance misuse, history of violence, no fixed abode after discharge and higher number of previous admissions, whereas older age and living with family or relatives were protective factors. A history of violence, HoNOS item 1, prescription of clozapine and conditional discharge were not statistically significant factors in the predictive model. There are several possible reasons why the predictors were different for the patient- and episode-level data. First, the sample size used for regression analysis for the patient-level data was about half that of the episode-level data. A smaller sample size might not have enough power to detect any statistically significant results. In the episode-level data analysis, the highest odds ratio for any single predictor was less than 1.5. Researchers have also found that the effects of several predictors changed over time, so what was once not significant may become important factors in subsequent episodes.Reference Tulloch, David and Thornicroft41 This is exemplified with HoNOS item 1, prescription of clozapine and conditional discharge not found to be statistically significant factors in the patient-level data. However, the finding of these three as significant results in the episode-level data sheds light on areas that may be potentially amenable and worthy of attention, particularly if the patient has any residual aggression or overactivity upon discharge, given that some other predictors are unmodifiable in nature. Finally, it is rather difficult to engage patients who have no fixed abode after discharge and it can be more difficult to deliver any transition arrangements or community interventions to them.
Limitations and future directions
This study focused on readmissions via the accident and emergency department, where there is a higher chance that the readmissions would be unplanned. On the other hand, readmissions may occur via out-patient clinics and may also be unplanned, though this would be hard to delineate in practice, as noted by the OECD data.12 This would therefore affect the sample of patients identified in the readmitted group. The data used in this study was obtained through CDARS, which is more limited in scope compared with a review of discharge summaries or patient interviews. The reliance on psychiatrist input has contributed to the vast amounts of missing data and different definitions of certain data points, such as the duration of illness.
Unlike other studies, this study has not incorporated any standardised tests into the prediction model simply because such instruments are not routinely used to assess the patients upon discharge, instead relying solely on HoNOS scores. HoNOS scores crudely reflect the patient's psychotic symptoms in its measurement and have been argued to exhibit only moderate interrater reliability in some literature.Reference Adams, Palmer, O'Brien and Crook53–Reference Brooks55 Furthermore, there have not been any studies on the validity and reliability of the use of HoNOS in Hong Kong and the duration of training on the use of HoNOS across the city is not unified, meaning that the contribution of HoNOS item 1 as a predictor variable should be further investigated. Other studies have tried to incorporate other rating instruments in their prediction model for unplanned readmission; however, inclusion did not improve the power of the model.Reference Vigod, Kurdyak, Seitz, Herrmann, Fung and Lin13,Reference Moss, Li, Tobin, Weinstein, Harimoto and Lanctôt44 In addition, the model has not adequately assessed the post-discharge social and health system variables for their contribution to readmissions. Data on the out-patient treatment of the patient, such as the time to follow-up, was not incorporated in this study, which may in turn affect readmission rates.Reference Marcus, Chuang, Ng-Mak and Olfson56 It is also likely that the differing practices of the various hospitals and attending psychiatrists will have affected the decision to admit a patient to hospital, thereby influencing the readmission rates.Reference Moss, Nauranga, Kim, Rosen, Wang and Lanctot57
There is therefore much work to be done in the future to look into other predictor variables, including the incorporation of social and system factors. One possible method would be the use of artificial intelligence and deep learning. Much of the free-text data in the discharge summaries were left out in the analysis. The backwards regression model also had its inherent flaws in removing predictors one by one, without considering how the dropped factors might interact with each other when other variables were dropped, and models with large sample sizes can easily drive the P-values of variables to be less than 0.05.Reference Urach, Zauner, Wahlbeck, Haaramo and Popper58 There has already been research published that has used deep learning methods to predict certain health outcomes, such as in-hospital mortality, by scanning through various electronic health record data and converting the information into data that could be analysed, achieving discriminative powers of up to 0.93–0.94 in the c-statistic, although the 30-day unplanned readmission model could only reach a c-statistic of 0.75–0.76.Reference Rajkomar, Oren, Chen, Dai, Hajaj and Liu59 Research that focused on predicting violence and suicide also noted that the discriminative power was vastly improved when machine learning approaches were adopted.Reference Walsh, Ribeiro and Franklin60,Reference Menger, Spruit, van Est, Nap and Scheepers61
Implications
Despite the debate on whether readmissions should be used as a proxy for the quality of care provided, many hospitals are still routinely assessed on readmission rates and are penalised if the rates are too high. The actual reasons why patients are rapidly readmitted may be influenced by a variety of patient-levels factors, broader social and environmental factors such as social support, and hospital- and health-system-level factors. As all of the predictors in this model can be easily generated without much manual effort, the system can calculate the risk of readmission for each patient. This automated work process is particularly important as it would require no additional clinician manipulation to calculate the risk of each patient and a concerted approach may then be adopted from prior to discharge transitioning to out-patient care.
This clinical risk model is the first study to predict unplanned readmissions within 28 days for patients with psychotic spectrum disorders. Both the derivation and validation model were able to achieve moderate discriminative ability, with a c-statistic of 0.705 and 0.684, respectively. With this predictive model, the risk of readmission for each patient can be calculated and those predicted to be at high risk may be identified. Adjustments can then be made to their discharge arrangements or treatment may be implemented, focusing on the risk factors noted in this study and other effective measures mentioned in the literature, to prevent this undesirable outcome.
Funding
K.H. has received financial assistance from Janssen through the Hong Kong Public Doctor's Association for attending a conference meeting.
Acknowledgements
We thank Professor Dinesh Bhugra for his comments on the article. Our gratitude also goes to Ms Emily Ying and Mr Jonathan Ip for their kind assistance with Microsoft Excel tutorials and Mr Jay Huang and Dr Eric Lau for their review of the SPSS analysis. Finally, we also thank Dr Jonathan Ng, Dr Gigi Man Chi Yu, Dr Fiona Yu, Dr Vincent Tang and Dr Lawrence Chi Lun Lai for acting as the principal investigators for the process of ethics application for each cluster within the hospital authority.
Data availability
The data related to this study is currently being stored at a safe location in accordance with Hospital Authority's policy on medical records.
Author contributions
K.H. was responsible for the design of the study, acquisition and analysis of the data and write-up of the study. K.M.C. was responsible for modifying the design of the study and critically revising the study write-up. J.L. was responsible for the interpretation of data and the critical revision of the study write-up. S.K.L. was responsible for conception of the study and the critical revision of the study write-up. S.S.Y.L. was responsible for the conception and design of the work and the critical revision of the study write-up.
Open access
eLetters
No eLetters have been published for this article.