Sample

The sample consisted of three groups of participants reporting anomalous experiences associated with psychosis: (a) individuals with a DSM–IV (American Psychiatric Association, 1994) diagnosis of a psychotic disorder (‘diagnosed group’ n=35), (b) help-seeking individuals meeting criteria for an ‘at risk mental state’ (n=21), and (c) individuals who had never received treatment for a diagnosis of psychotic disorder but had at least occasional experiences of any Schneiderian first-rank symptom (‘undiagnosed group’ n=35). Table 1 summarises the demographic features of the three groups.

The diagnosed group included 14 people recruited from an in-patient unit and linked community team specialising in the treatment of people with a first or second episode of psychosis (Lambeth Early Onset Team, LEO), and 21 people recruited via a specialist tertiary service providing psychological interventions for out-patients with psychosis (Psychological Intervention Clinic for Outpatients with Psychosis; PICuP), both in the South London and Maudsley Trust, UK.

The group with at-risk mental state was recruited through Outreach and Support in South London (OASIS), also based in the South London and Maudsley Trust, a clinical service for people meeting the Personal Assessment and Crisis Evaluation Clinic (PACE) criteria for an at-risk mental state (Reference Phillips, Yung and McGorryPhillips et al, 2000).

The undiagnosed group was recruited from multiple sources through advertisement to obtain a sample with as much variety in socio-economic and cultural background as possible. All volunteers were screened for suitability with a self-report questionnaire which enquired about lifetime incidence of the range of anomalous experiences that make up the inventory section of the AANEX. Only individuals with at least occasional experiences of any Schneiderian first-rank symptom which was not directly related to drug use and occurred under conditions of clear consciousness were invited to participate. Only those whose anomalous experiences had commenced more than 5 years previously but who had never been treated or sought clinical help were included. This was to distinguish this group from those with at-risk mental state by minimising the likelihood that they would develop a need for care relative to these experiences in the future.

Exclusion criteria for all three groups were inability to speak fluent English, a history of neurological problems, head injury or epilepsy, evidence of current substance dependence and estimated current IQ<70.

The undiagnosed and diagnosed groups did not differ significantly significantly in age; however, those with at-risk mental state were significantly younger than the undiagnosed group (mean difference 0.362, 95% 0.201–0.524, P<0.001), as a result of the selection criteria (i.e. anomalous experiences having been present for at least 5 years). There were no group differences in gender ratio or ratio of British Whites to other ethnic categories. There were significant group differences in estimated IQ (Bonferroni corrected), with the undiagnosed group having a higher mean IQ than either of the clinical groups (undiagnosed>diagnosed, mean difference 26.4, 95% CI 10.2–42.6; P<0.001; undiagnosed>at-risk mental state, mean difference 16.9, 95% CI 33.3–0.48, P=0.04).

Development of the AANEX

The AANEX was developed in sequential stages, beginning with exploratory in-depth interviews. Initially, a preliminary interview schedule was created to explore the variables suggested to be pertinent by the cognitive models (e.g. Reference Fowler, Birchwood, Fowler and JacksonFowler, 2000; Reference Garety, Kuipers and FowlerGarety et al, 2001) and to gather information that might suggest additional dimensions of interest. Interviews were then carried out with six participants with a history of anomalous experiences, three of whom had been treated for a psychotic disorder and three of whom had not.

On the basis of the information gained from the preliminary interviews, a more structured interview schedule was developed. The in-depth interviews demonstrated that people's appraisals and responses do not remain static but change over time. For this reason, a format was developed that enabled assessment of more than one time period in an individual's life, including the first onset and current experiences, and also other intermediate time periods. Sets of items can be repeated to yield scores for each relevant time period and/or each particular form of anomalous experience that is reported, giving different ‘response sets’.

The next stage involved the piloting of this interview schedule on 10 participants, including individuals diagnosed with a psychotic disorder, those considered to be at risk of psychosis and those who had not received a diagnosis.

All elements of the interview schedule were tested repeatedly in a process of iterative modification and trial, until a final version was reached, which only incorporated useful and reliable questions that yielded easily and usefully codable responses. Particular attention was paid to establishing a flexible structure that would allow responses of different groups to specific experiences to be compared. The development of the scoring scheme, using ordinal (rather than dichotomous) ratings and open-ended questions, aimed to reflect and capture the continua of belief, feelings and responses expressed by the participants.

A cross-sectional pilot comparison was then carried out between a sample of undiagnosed participants (n=8) and diagnosed participants (n=8), using the final version of the interview. Results included two trends towards significant differences in types of appraisal between the two groups, despite the small samples. The interview was acceptable to all participants. Copies of the AANEX Interview and the users' manual are available from C.M.C.B.

Components of the AANEX

Procedure

Participants were administered the AANEX as part of a wider battery of tests. Interviews were carried out in person, and were tape-recorded with the consent of the participants. Initial ratings made at the time of the interview were checked on the basis of the taped interviews.

A subset of participants (n=9) were administered the AANEX over two sessions (no more than a week apart) when the interview was too long or tiring to be completed in a single sitting. The administration time varied from 45 to 300 min (mean 130 minutes), depending upon the individual's style of response, and the number of different anomalies and time points to which the interview questions were anchored. The number of response sets ranged from 1 to 18, with a mean of 3.7 per person. All participants were paid an honorarium for their time.

Statistical analyses

Since all of the variables being tested were ordinal (either rated between 0 and 2 or 1 and 5), the group comparisons were made using multinomial logistic regression analyses (Reference Tabachnick and FidellTabachnick & Fidell, 2001). The responses of the three groups were pooled and group was entered as two dummy variables, with the undiagnosed group as the reference category. P values of predictor variables were based on the Wald statistic. Since analysis of the AANEX data needed to account for the fact that repeated observations within participants (i.e. appraisals of different kinds of anomalies or at different time points) were not independent, robust standard errors were used to account for the correlation between repeated observations from the same participant. The group scores for the interview variables in Table 3 have been summarised using means and standard deviations, treating the scales as interval data for this purpose only. Statistical analyses were conducted using STATA version 8.1.

Interrater reliability

Interrater reliability was assessed by comparing ratings from C.M.C.B. with those from two independent raters (L.C.J. and E.P.P.) who are chartered clinical psychologists with clinical and research experience of assessing psychotic phenomena. Each rated four interviews. Combined, the eight interviews yielded eight sets of inventory ratings and 36 response sets on the AANEX–CAR, since each interview comprised several sets of responses anchored to different time points in the respondent's life or different kinds of experiences.

The raters scored the interviews from tapes on the basis of the written scoring manuals following a brief training session. This involved both raters initially being given the same transcribed interview to score. Their scores were checked and any areas of concern were discussed and clarified. The raters were masked to the diagnostic status of the participants.

The degree of agreement between C.M.C.B. and those of the independent raters (treated as one data-set) was analysed pairwise using weighted kappa. The weight matrices used for the comparison of ratings were determined for each variable, taking into account the rating scale, whether it was ordinal and the relative intervals between each anchor on the scale.

AANEX–inventory

The average kappa for all 40 items was 0.67, which can be interpreted as substantial agreement; 92.5% of the items had at least fair agreement (>0.4), 42.5% had substantial agreement (>0.6) and 17.5% had almost perfect agreement (>0.8).

Of the 40 items, receptivity, reference experiences and thought pressure achieved weighted kappa <0.4. The raw percentage agreements for these three items were 62%, 79% and 54% respectively. The reference experiences items ranged between 3 and 5 in the data-set compared; combined with the substantial raw agreement, this suggests that the low kappa value might have reflected the incomplete range of scores being represented. However, the receptivity and thought pressure scores in the data-set ranged between 1 and 5 and 1 and 4 respectively, suggesting that the kappa value was not overly conservative.

Subsequently, the scoring guidelines were altered to facilitate the rating of these items. Further assessment is necessary to establish the interrater reliability of the revised version. The data reported here were collected by C.M.C.B. only.

AANEX–CAR

The level of agreement for the variables assessing the categories of appraisal, dimensions of appraisal (valence, dangerousness, externality and agency), emotional response, context and implications did not fall below 0.4; 65% had at least substantial agreement and 35% almost perfect agreement.

The subsection assessing cognitive and behavioural response to anomalies contained three items that did not achieve satisfactory interrater reliability: reappraisal, rumination and neutral response. Of these, the reappraisal item showed raw agreement of over 80%, whereas the latter two items showed raw agreement of 66% and 58% respectively, suggesting that amendments are required to either the scoring guidelines or the categorisation of responses to this section. The scoring guidelines have subsequently been amended and further work is needed to establish the current reliability of these rating categories. The other three categories in this section (avoidance, cognitive control and immersion) demonstrated fair to substantial agreement. Data from the rumination and neutral response categories have not been included in the analyses currently reported. Future data generated by the probe item may be analysed to identify alternative schemes of categorisation.

Validity

Content validity was addressed by developing items on the basis of both the psychological literature and a range of existing clinical measures, as well as in-depth interviews with a range of individuals having anomalous experiences. It was not feasible to assess concurrent validity since no single existing instrument measures the same variables. The cross-sectional study reported here assesses the construct validity of the AANEX by exploring whether the interview can distinguish those with and without a need for care in the context of psychotic-like experiences. Specifically, based on the model of Garety et al (Reference Garety, Kuipers and Fowler2001), good construct validity would be demonstrated by:

1. (a) emotional response (the diagnosed and at-risk mental state groups reporting greater emotional distress and arousal in relation to anomalies than the undiagnosed group;

2. (b) dimensions of appraisals (the diagnosed and at-risk mental state groups appraising anomalies as more negative, dangerous and with more external and personal causes than the undiagnosed group;

3. (c) categories of appraisal (the diagnosed group making more externalising appraisals and less internalising appraisals than the undiagnosed group);

4. (d) perceived controllability and social support/understanding (the undiagnosed group reporting higher perceived controllability and higher perceived social support pertaining to their experiences than the diagnosed or at-risk mental state groups);

5. (e) cognitive and behavioural responses (although no specific cognitive and behavioural responses to anomalies are predicted by the model of Garety et al (Reference Garety, Kuipers and Fowler2001), it might be expected that different patterns of response would be reported by the undiagnosed group compared with the other groups.

Therefore, cross-sectional analyses relevant to these predictions are reported, based on a subset of the full data generated by the AANEX.

Group comparisons

Table 3 summarises the item scores for the three groups and the results of the regression analyses. The ‘state’ component scores were entered as covariates in all the analyses. The following sections report the analyses carried out to test construct validity via the predictions of the model of Garety et al (Reference Garety, Kuipers and Fowler2001).

Limitations

There are some limitations to the generalisability of these preliminary findings, because of the nature of the undiagnosed sample. Relative to the clinical groups, the undiagnosed participants were more likely to come from White ethnic backgrounds (although there were more non-British people), which may be relevant to their styles of response and appraisal, as well as their social context. They also differed significantly from the diagnosed group in terms of estimated IQ. Although this may not affect appraisal processes per se, it might represent an additional factor influencing the development of need for care: the undiagnosed group had higher than average IQ, which might possibly act as a protective factor through allowing more sophisticated appraisals and responses. However, it is not known whether this is characteristic of all those in the general population with at least occasional experiences of any first-rank symptom.

Furthermore, although the number and types of anomalies occurring at each time point were controlled for, the frequency or severity of any particular type was not. It is likely that variance in frequency has an impact on appraisals and response. Nevertheless, the undiagnosed group was selected on the basis of reporting comparable anomalous experiences to the clinical groups, and individuals reporting only infrequent experiences were not included.

Implications

Overall, the initial results suggest that the AANEX has the potential to elicit information that may clarify the nature of the continuum of psychotic and psychotic-like experiences, and the complex and multifactorial development of distress and need for care relative to these experiences. The interview may allow this clarification by assessing a range of components of experience considered to vary across the continuum, such as types and frequency of anomalous experience, interpretations, and emotional and contextual factors. In relation to this, it has demonstrated utility as a clinical tool for differentiating anomalous experiences from their appraisals, thereby improving understanding of the person's subjective experience and how they might benefit from psychological interventions.

The majority of the interview schedule was found to have good levels of reliability, indicating that the dimensions assessed and scoring schemes are robust, despite the complexity of the material being elicited. The small number of items that required amendment have been refined on the basis of the data gathered in this study, aiming for clearer, more unidimensional definition.

The form and process of the interview was acceptable to participants from both clinical and non-clinical populations, and sensitive to the differing ways in which psychotic-like experiences may be interpreted. These characteristics differentiate the AANEX from existing symptom rating scales that assess only pre-specified ‘clinical’ forms of anomalous experiences (e.g. the Scale for Assessment of Positive Symptoms (Reference AndreasenAndreasen, 1984) or the Positive and Negative Syndrome Scale; Reference Kay, Fiszbein and OplerKay et al, 1987), and also from other scales suitable for populations without psychosis that do not assess relevant context, appraisal and response variables (e.g. the CAARMS (Reference YungYung, 2000) or WMAPE (Reference Kwapil, Chapman and ChapmanKwapil et al, 1999)). The flexible structure of the AANEX facilitates its use for either detailed investigations of individuals' experiences or briefer assessments that may be repeated at several time points (e.g. pre- and post-intervention or during a follow-up period). The brief form of the AANEX is currently being piloted as a repeated assessment during a follow-up of clients of OASIS with at-risk mental state.