Study population and inclusion/exclusion criteria

A multicentre, case-register study was conducted to examine the correlations between the short versions of the PANSS, YMRS and MADRS scales and their proxy versions. The sample was recruited from patients aged 18–55 years attending the mental health services (emergency, in-patient and out-patient settings) of 12 public hospitals in Andalusia (southern Spain) for treatment of first-episode psychosis (FEP). In addition, an independent cohort of patients from the Treatment and Early Intervention in Psychosis Program (TIPP-Lausanne; Switzerland) was used for external validation of the proxy instruments.^{Reference Baumann, Crespi, Marion-Veyron, Solida, Thonney and Favrod21} The sum of all of these hospitals comprised an epidemiological catchment area of 5.5 million (approximately 3 million aged 18–55 years). Psychosis was confirmed with the Mini-International Neuropsychiatric Interview in the primary cohort and the Comprehensive Assessment of At Risk Mental States in the TIPP-Lausanne.^{Reference Sheehan D, Lecrubier, Sheehan, Amorim, Janavs and Weiller22,Reference Yung, Yuen, McGorry, Phillips, Kelly and Dell'Olio23} All patients met the ICD-10 diagnostic criteria for psychotic disorders (F1x.4, F1x.5, F1x.7, F20–29, F30.2, F31.2, F31.5, F31.6, F32.3, F33.3 and F53.1 codes).²⁴ Exclusion criteria were organic psychoses, past diagnosis of psychotic disorder, severe or unstable medical condition, history of traumatic brain injury, cognitive impairment or major neurological disease, or any level of intellectual disability or autism spectrum disorder. The Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) statement guidelines were followed (see checklist in Supplementary Table 1 available at https://doi.org/10.1192/bjo.2023.623 in Supplementary Appendix 1).^{Reference Von Elm, Altman, Egger, Pocock, Gøtzsche and Vandenbrouckef25} The authors assert that all procedures contributing to this work comply with the ethical standards of the relevant national and institutional committees on human experimentation and with the Helsinki Declaration of 1975, as revised in 2008. Patient consent was obtained in accordance with the requirements of the Andalusian Biomedical Research Ethics Committee (approval number 2184-N-21; informed consent was not required because of the use of de-identified data) and the Human Research Ethics Committee of the Canton Vaud (protocol number 2020-00272; verbal consent was witnessed and formally recorded). All procedures involving human participants in this research were approved by both ethics committees.

Selected demographic, clinical and psychometric variables

Information on patient sociodemographic, clinical and psychometric characteristics was acquired at study enrolment by a clinician-administered questionnaire designed by the authors. Sociodemographic data collected included age, gender, ethnicity (European White or other), marital status (married/partnership or unmarried), living situation (alone or with others), education (higher education and secondary/lower education) and occupation (employed, unemployed or student). Clinical characteristics collected included age at onset of psychosis, duration of untreated psychosis (following the methodology outlined in the Nottingham Onset Schedule^{Reference Singh, Cooper, Fisher, Tarrant, Lloyd and Banjo26}), first-degree family history of psychosis, premorbid psychosocial adjustment (assessed with a validated proxy measure to discriminate patients with from those without poor premorbid adjustment^{Reference López-Díaz, Ayesa-Arriola, Garrido-Torres, Otíz-García de la Foz, Suárez-Pinilla and Ramírez-Bonilla20}), psychiatric history, history of harmful or hazardous substance use (alcohol, cannabis or other illicit drugs), type of psychotic disorder (according to ICD-10 coding), presence of suicidal symptoms (including suicidal ideation, plans or attempts according to the Paykel Suicide Scale methodology^{Reference Silverman, Berman, Sanddal, O'Carroll and Joiner27–Reference Paykel, Myers, Lindenthal and Tanner29}) and patient status (in-patient v. out-patient) at the time of enrolment. Psychometric measures were assessed at time of enrolment, using the shortened versions of the PANSS, YMRS and MADRS and their respective proxies.^{Reference Østergaard, Lemming, Mors, Correll and Bech30–Reference Tohen, Katagiri, Fujikoshi and Kanba32} The time of the psychometric assessment (acute or stabilisation phase) and the experience of the clinician assessing the proxies (less than or over 5 years) were also recorded.

Abbreviated versions of PANSS, YMRS and MDRS scales

The six-item version of the PANSS (PANSS-6) is a brief rating scale focusing on the severity of key schizophrenia symptoms. It consists of three items each for the positive and negative subscales (P1 delusions, P2 conceptual disorganisation, P3 hallucinations, N1 blunted affect, N4 social withdrawal, N6 lack of spontaneity and flow of conversation). PANSS-6 is psychometrically valid, reliable, sensitive to change, scalable and can be used to define remission in patients with psychosis.^{Reference Østergaard, Lemming, Mors, Correll and Bech30} Symptomatic remission is defined as a PANSS-6 total score <14 and a score of mild or less (≤3) on each of its six items.^{Reference Østergaard, Lemming, Mors, Correll and Bech30,Reference Andreasen, Carpenter, Kane, Lasser, Marder and Weinberger33}

The abbreviated version of the YMRS is used to evaluate the six core clinical symptoms of a manic episode: elevated mood (item 1), increased motor activity/energy (item 2), decreased need for sleep (item 4), increased rate or amount of speech (item 6), language/thought disorder (racing thoughts or flight of ideas) (item 7) and insight (excessive involvement in risky activities) (item 11).^{Reference Tohen, Kanba, McIntyre, Fujikoshi and Katagiri31} The YMRS-6 is a brief instrument that measures treatment response in manic and mixed episodes adequately. Clinical remission is defined as a score of <4.^{Reference Tohen, Kanba, McIntyre, Fujikoshi and Katagiri31,Reference Berk, Ng, Wang, Calabrese, Mitchell and Malhi34}

The MADRS-6 is a six-item subscale score of the original test, widely acknowledged to focus on the core symptoms of depression: apparent sadness (item 1), reported sadness (item 2), inner tension (item 3), lassitude (item 7), inability to feel (item 8) and pessimistic thoughts (item 9).^{Reference Tohen, Katagiri, Fujikoshi and Kanba32} The MADRS-6 is a reliable instrument for assessing antidepressant efficacy in affective disorders. A score of <5 points is considered clinical remission.^{Reference Tohen, Katagiri, Fujikoshi and Kanba32,Reference Berk, Ng, Wang, Calabrese, Mitchell and Malhi34}

Proxy tools for the assessment of the PANSS-6, YMRS-6 and MADRS-6, and methodology for measuring in EHR-based studies

For scoring the PANSS-6, YMRS-6 and MADRS-6 proxies, clinicians carefully read the patient's EHR considering the setting in which the clinical note was written (emergency unit, in-patient admission, community follow-up, etc.). Patients with non-affective psychosis were assessed with the PANSS-6 proxy, and those with affective psychosis were also assessed with the YMRS-6 proxy and/or the MADRS-6 proxy, depending on whether it was a case of mania, depression or mixed episodes with psychotic symptoms. When symptoms were not reported as such in the medical record (relatively commonplace in routine practice because of time constraints during visits), clinicians had to complete the missing data by contextualising them based on the clinical setting (emergency department, in-patient admission, community follow-up, etc.), their experience and knowledge of the psychotic disorder being assessed.

The proxies and their scoring are presented in Fig. 1. Proxies for psychotic and affective features were constructed with a symptom checklist based on items from the PANSS-6, YMRS-6 and MADRS-6. Symptoms were classified as absent/mild, present or severe according to the following criteria: (a) ‘absent/mild’ when symptoms are absent or at the upper end of normal limits or are mild and intermittent; (b) ‘present’ when symptoms are relatively stable and cause moderate distress and affect the patient; and (c) ‘severe’ when symptoms significantly interfere with functioning, interpersonal relationships and/or may seriously compromise the safety of the patient or others. The PANSS-6 proxy scores were 1 (absent/mild), 3 (present) and 6 (severe), with total scores ranging from 6 to 36 points. Clinical remission for the proxy PANSS-6 was defined as a score of 6, with all symptoms classified as absent/mild. The YMRS-6 and MADRS-6 proxies were scored as 0 (absent/mild), 3 (present) and 6 (severe), with total scores ranging from 0 to 36 points. For these proxies, a score of ≤3 was defined as clinical remission, with all but one of the symptoms classified as absent or mild. An example of the scoring method is shown in Fig. 2.

Fig. 1 The proxies and their scoring criteria.

Fig. 2 Example of proxy scoring method. CT, computed tomography; PANSS-6, Positive and Negative Syndrome Scale.

Standardised procedure for the validation of PANSS-6, YMRS-6 and MADRS-6 proxy measures

The proxy measures were validated in the context of real-world practice with patients with FEP. Clinicians participating in the study were trained in the use of the assessment tools and followed a manual with instructions designed by the first author. Thus, during a clinical interview, a first clinician recorded a clinical note describing the patient's mental condition, and a second clinician completed the psychometric assessment with the PANSS-6, YMRS-6 and/or MADRS-6 as appropriate, depending on whether the patient had a non-affective or affective psychosis. A third clinician, not present during the clinical interview and blinded to the scale scores, then carefully read the clinical notes in the patient's EHR and examined the severity of symptoms with the proxy tools. The concordance between the scale and proxy scores, using a variety of statistical methods, determined the validity of the proxy instruments.

DIRAYA: the electronic health information system of the Andalusian Health Service

The main cohort's EHR data were collected from DIRAYA, the digital health record software used in the Andalusian public health system (Spain). The DIRAYA is an EHR system that integrates all of the health information for each patient into a single regional record linking all of the clinical data from primary care centres, specialist out-patient clinics, hospitals, emergency services and pharmacies, including electronic drug prescriptions, telemedicine, referrals, radiology, laboratory tests, appointments, etc. The DIRAYA is one of the largest registration systems in Europe, containing clinical information on the more than 8 million people in the Andalusian public health network, and is therefore a very valuable resource for epidemiological studies.^{Reference Sánchez-Teruel, Robles-Bello and Sarhani-Robles35,Reference Protti36}

Data analysis

Descriptive statistics for patient characteristics were expressed as percentages, mean or median, and s.d. or interquartile range (IQR), as appropriate. Normal distribution was tested with the Kolmogorov–Smirnov test. Differences between PANSS-6, YMRS-6 and MADRS-6 and their respective proxy versions were examined by univariate and multivariate analyses. An a priori power analysis was performed to calculate the appropriate sample size. With an expected correlation coefficient of 0.6, an assumed type 1 error of 0.05 and a type 2 error rate of 0.20, each of the proxies required a minimum sample size of 19 to achieve adequate statistical power. Either the Pearson's or Spearman's coefficient was used for binary correlations between scores on each instrument, depending on whether or not the normality criteria were met. Partial correlation coefficients were performed to estimate the relationships between the psychometric scales and proxy scores, controlling for potential confounders such as type of psychosis (non-affective or affective), time of assessment (acute or stable) and clinician experience in scoring the proxies (less than or over 5 years). Scatter plots were drawn to display the correlations between variables. The extent to which the proxies predicted clinical remission was determined by regression analysis (using the above cut-off points for each scale and its corresponding proxy). Odds ratios and 95% confidence intervals were estimated by logistic regression. The Nagelkerke R ² was used to assess the variance explained by the model. A classification table and area under the receiver operating characteristic curve (AUC) were used to evaluate the accuracy and discriminatory ability of the proxies. Internal validation was performed with bootstrapping (2000 resamples). External validation was performed by repeating the same statistical analyses described above in the TIPP-Lausanne cohort patients with early psychosis. Thus, the replicability of the findings in a different data-set would ensure the generalisability of the proxy tools. Interrater reliability of the proxy method was tested by intraclass correlation coefficient (ICC), two-way random effects model and absolute agreement. To this end, one of the participating patients diagnosed with a psychotic episode with mixed affective features was selected at random to assess the interrater reliability of the PANSS-6, YMRS-6 and MADRS-6 proxies, using a single case history. The ratings of this patient's clinical notes were examined by 20 different raters across all of the participating centres for ICC calculation. Missing data were managed with the listwise deletion method. Significance was set at P < 0.05. All statistical analyses were conducted with SPSS Statistics version 24 for MacOS (IBM Corporation, Armonk, New York, USA) and MedCalc Statistical Software version 20 for Mac OS (MedCalc Software, Ostend, Belgium; see https://www.medcalc.org).