INTRODUCTION
Cognitive dysfunction is an enduring feature of schizophrenia associated with low levels of occupation attainment, social engagement, and residential independence, even in the context of clinical remission (Bowie et al., Reference Bowie, Leung, Reichenberg, McClure, Patterson, Heaton and Harvey2008; Bowie, Reichenberg, Patterson, Heaton, & Harvey, Reference Bowie, Reichenberg, Patterson, Heaton and Harvey2006; Fett et al., Reference Fett, Viechtbauer, Dominguez, Penn, van Os and Krabbendam2011; Green, Kern, & Heaton, Reference Green, Kern and Heaton2004; Harvey & Strassnig, Reference Harvey and Strassnig2012; Rossell & David, Reference Rossell and David1997). Appreciation of cognition as a strong contributor to psychosocial disability has led to the development of interventions specifically designed to target cognitive impairment. Cognitive remediation (CR) therapies are viewed as evidence-based methods capable of alleviating cognitive deficits in schizophrenia (Bryce, Sloan, Lee, Ponsford, & Rossell, Reference Bryce, Sloan, Lee, Ponsford and Rossell2016; Grynszpan et al., Reference Grynszpan, Perbal, Pelissolo, Fossati, Jouvent, Dubal and Perez-Diaz2011; McGurk, Twamley, Sitzer, McHugo, & Mueser, Reference McGurk, Twamley, Sitzer, McHugo and Mueser2007; Wykes, Huddy, Cella, McGurk, & Czobor, Reference Wykes, Huddy, Cella, McGurk and Czobor2011), and consequently, are being considered for use in mental health rehabilitation (Galletly et al., Reference Galletly, Castle, Dark, Humberstone, Jablensky, Killackey and Tran2016). Nevertheless, there are limitations in studies to date that impact the strength of recommendations for clinical translation.
First, there have been few rigorous efficacy trials (Dixon et al., Reference Dixon, Dickerson, Bellack, Bennett, Dickinson, Goldberg and Kreyenbuhl2010). Only one-third of controlled studies met criteria for “good methodology” (i.e., 13/40) in one of the most comprehensive reviews of CR (Wykes et al., Reference Wykes, Huddy, Cella, McGurk and Czobor2011). Some studies using active controls and blinded assessors have demonstrated poor transfer to untrained cognitive tasks following therapy (Dickinson et al., Reference Dickinson, Tenhula, Morris, Brown, Peer, Spencer and Bellack2010; Gomar et al., Reference Gomar, Valls, Radua, Mareca, Tristany, del Olmo and McKenna2015). Additionally, less than a third of controlled CR trials include follow-up assessments to examine sustainability of change (Paquin, Wilson, Cellard, Lecomte, & Potvin, Reference Paquin, Wilson, Cellard, Lecomte and Potvin2014; Wykes et al., Reference Wykes, Huddy, Cella, McGurk and Czobor2011).
Trial quality and its impact on cognitive and functional enhancement in CR research, as well as potential changes in the proportion of high quality studies being produced, has not been systematically investigated since the 2011 meta-analysis by Wykes and colleagues. Nevertheless, many recently published CR reports lack comparison groups (Murthy et al., Reference Murthy, Mahncke, Wexler, Maruff, Inamdar, Zucchetto and Alexander2012; Pillet et al., Reference Pillet, Morvan, Todd, Franck, Duboc, Grosz and Amado2015; Sharip et al., Reference Sharip, Michie, Schall, Drysdale, Case, Sankaranarayanan and Das2013) or primarily use treatment-as-usual controls (d’Amato et al., Reference d’Amato, Bation, Cochet, Jalenques, Galland, Giraud-Baro and Brunelin2011; Hargreaves et al., Reference Hargreaves, Dillon, Anderson-Schmidt, Corvin, Fitzmaurice, Castorina and Donohoe2015; Royer et al., Reference Royer, Grosselin, Bellot, Pellet, Billard, Lang and Massoubre2012). These trials also inconsistently explore whether training-related improvements endure over time. Methodological limitations remain an area of ongoing discussion in the CR literature (McGurk et al., Reference McGurk, Mueser, Covell, Cicerone, Drake, Silverstein and Essock2013).
Second, CR interventions are often embedded within broader therapeutic programs, which can include other psychological or vocational interventions that impact treatment outcome(s) (Bowie, McGurk, Mausbach, Patterson, & Harvey, Reference Bowie, McGurk, Mausbach, Patterson and Harvey2012; McGurk, Mueser, & Pascaris, Reference McGurk, Mueser and Pascaris2005; Vauth et al., Reference Vauth, Corrigan, Clauss, Dietl, Dreher-Rudolph, Stieglitz and Vater2005). Whilst combined approaches appear essential for optimizing functioning (Bowie et al., Reference Bowie, McGurk, Mausbach, Patterson and Harvey2012; Wykes et al., Reference Wykes, Huddy, Cella, McGurk and Czobor2011), non-specific effects derived from alternative therapies, such as social contact or therapeutic engagement, may account for some of the observed gains in treatment trials (Kurtz, Seltzer, Shagan, Thime, & Wexler, Reference Kurtz, Seltzer, Shagan, Thime and Wexler2007; McGurk et al., Reference McGurk, Mueser, Covell, Cicerone, Drake, Silverstein and Essock2013).
Reviews that exclude studies integrating CR with vocational training provide less compelling support for the impact of CR on measures of cognition and psychosocial functioning relative to controls (NICE, 2009, 2014). Combining CR with strategy compensation (i.e., “drill-and-strategy” training) may represent an alternative means of supporting functional generalization in the absence of additional therapeutic programs; however, little is known about the extent to which these impact treatment success (McGurk et al., Reference McGurk, Mueser, Covell, Cicerone, Drake, Silverstein and Essock2013; Penades et al., Reference Penades, Catalan, Pujol, Masana, Garcia-Rizo and Bernardo2012).
Finally, the feasibility of community-based CR is a potential translational challenge. Of particular concern, some interventions demonstrating large effect sizes require 40–100 hr of training and high treatment intensities (e.g., 5 days/week) (Fisher, Holland, Merzenich, & Vinogradov, Reference Fisher, Holland, Merzenich and Vinogradov2009; Fisher, Holland, Subramaniam, & Vinogradov, Reference Fisher, Holland, Subramaniam and Vinogradov2010; Penades et al., Reference Penades, Catalan, Salamero, Boget, Puig, Guarch and Gasto2006). This may be problematic clinically as sustaining engagement over lengthy intervention periods is a potential barrier to psychosocial treatment use in schizophrenia (Parker, Foley, Walker, & Dark, Reference Parker, Foley, Walker and Dark2013).
There has been limited consideration of treatment outcomes beyond cognition, symptoms, and functional capacity or real-world behavior(s) in CR trials (Kern, Glynn, Horan, & Marder, Reference Kern, Glynn, Horan and Marder2009; Kurtz, Reference Kurtz2012). There is growing interest in the role of self-efficacy—beliefs regarding one’s confidence of achieving success on a specific task (Bandura, Reference Bandura1977)—as a potential therapeutic target and process variable in cognitively enhancing treatment (Wykes & Reeder, Reference Wykes and Reeder2005; Wykes & Spaulding, Reference Wykes and Spaulding2011). CR purports to target cognitive skills underlying real-world functioning (Bowie et al., Reference Bowie, Leung, Reichenberg, McClure, Patterson, Heaton and Harvey2008; Wykes et al., Reference Wykes, Reeder, Huddy, Taylor, Wood, Ghirasim and Landau2012), but people with schizophrenia are also likely to have poor confidence in their ability to complete everyday tasks if they have experienced failure and low functional achievement (Medalia & Richardson, Reference Medalia and Richardson2005).
Self-efficacy has been associated with daily functioning in schizophrenia through its relationships with cognition, negative symptoms and functional capacity (Cardenas et al., Reference Cardenas, Abel, Bowie, Tiznado, Depp, Patterson and Mausbach2013; Pratt, Mueser, Smith, & Lu, Reference Pratt, Mueser, Smith and Lu2005; Ventura et al., Reference Ventura, Subotnik, Ered, Gretchen-Doorly, Hellemann, Vaskinn and Nuechterlein2014). Nevertheless, while perceived competency in performing challenging cognitive exercises appears to improve following CR (Bowie, Grossman, Gupta, Holshausen, & Best, Reference Bowie, Grossman, Gupta, Holshausen and Best2017; Choi & Medalia, Reference Choi and Medalia2010), there has been little investigation of whether self-efficacy concerning everyday living and social behaviors is specifically responsive to this type of intervention. This is despite self-efficacy representing an important component of recovery.
While infrequently examined, everyday self-efficacy has the potential to have a broader impact on community functions in schizophrenia (Cardenas et al., Reference Cardenas, Abel, Bowie, Tiznado, Depp, Patterson and Mausbach2013). It could be responsive to cognitive enhancing interventions if participants learn strategies of functional relevance, have positively reinforcing learning experiences, achieve task mastery, observe peers succeeding on challenging tasks, and receive support as well as attributional and performance-based feedback (Bandura, Reference Bandura1977; Medalia & Choi, Reference Medalia and Choi2010; Wykes & Reeder, Reference Wykes and Reeder2005).
Aims of the Study
This study aimed to evaluate the impact of a 20-session, computer-assisted, drill-and-strategy CR intervention on cognitive performance, everyday self-efficacy, and independent living skills in people with schizophrenia. The intervention was delivered in a group setting in the community and compared directly with a computer game (CG) playing condition. It was hypothesized that CR would produce greater improvements in global cognition, self-efficacy and independent living skills than the active control condition.
METHODS
Design and Participants
The trial was an assessor-blind randomized controlled trial with two independent interventions delivered in parallel. The study was approved by hospital and university human research ethics committees and reported in accordance with CONSORT guidelines (Schulz, Altman, Moher, & the CONSORT group, Reference Schulz, Altman and Moher2010). Data described in this report were obtained in compliance with the Helsinki Declaration.
Individuals diagnosed with schizophrenia or schizoaffective disorder (American Psychiatric Association, 2000) were recruited from Melbourne-based mental health services in Victoria, Australia. Recruitment sites included community-based clinics or support services and residential rehabilitation programs. Participants (or their significant other) either contacted the research team directly in response to advertisement or staff involved in their care informed the team of potentially interested individuals made aware of the study aims.
Diagnoses were confirmed using the Mini International Neuropsychiatric Interview (v.5.0; Sheehan et al., Reference Sheehan, Lecrubier, Sheehan, Amorim, Janavs, Weiller and Dunbar1998) and reviews of available clinical documentation. All participants gave written informed consent and were financially reimbursed for assessments only ($40 AUD for baseline assessment; $25 AUD each for end-assessment and follow-up). Recruitment and intervention groups were conducted between February 2015 and December 2016, with follow-up data collection extending until March 2017.
This study was designed to recruit 56 participants (28 per group). Factoring approximately 20% attrition, this would enable 44 to be included in the analysis, which would be sufficient to detect an anticipated medium effect (f=0.25) for the interaction term using a 2 × 3 factor design with power of .95 and an alpha of .05. Participants were eligible if they had a diagnosis of schizophrenia or schizoaffective disorder, were aged 18–65, and had sufficient English language skills to communicate with the research team. Exclusion criteria included: (1) psychiatric hospitalization in the previous 2 months (to infer potential clinical stability); (2) estimated intellectual disability (scaled score <70 on the Wechsler Test of Adult Reading); (3) severe neurological or sensorimotor impairment producing a sustained impact on cognition; (4) substance dependence or electroconvulsive therapy in the previous 6 months; and (5) concurrent participation in another interventional research trial.
Following recruitment, consent and baseline assessment, participants were assigned to a group determined by a randomized sequence of CR or CG playing created using a random number generator. Randomization followed a block-based schedule with between and within-block variability. A researcher (E.T.) independent of recruitment, intervention delivery, and data analysis managed this process.
Fifty-eight participants were recruited. Two participants were excluded at baseline due to intellectual disability and poor English language skills, resulting in a total randomized sample of 56 individuals. Of these participants, 98% were taking psychotropic medication, while 21% met MINI criteria for a depressive disorder, 20% met criteria for at least one anxiety disorder, and 9% met criteria for alcohol or other drug (cannabis or methamphetamine) abuse. Participants estimated as “borderline ill or less”, “mildly ill”, or “moderately ill or higher” was 33.3%, 37.1%, and 29.6%, respectively, based on PANSS total symptom scores (Leucht, Reference Leucht2014). Participant characteristics are summarized in Table 1. No significant between-group differences were observed for any variable in the eligible baseline sample (N=56); however, there was a statistical trend for higher self-efficacy in controls (p=.065). Participant progression through the study is illustrated in Figure 1.
Fig. 1 CONSORT flow diagram illustrating participant progression through the trial. CR=cognitive remediation; CG=computer games; FU=follow-up.
Table 1 Comparison of baseline demographic, clinical, and treatment characteristics for participants randomized to CR and CG
Note. a N=26;
b N=27;
c N=28.
CR=Cognitive remediation; CG=computer games; CPZE=Chloropromazine equivalence (see Woods, Reference Woods2003); EUROHIS-QOL=European Health Interview Survey – Quality of Life; ILSS-SR=Independent Living Skills Survey – Self-Report; M=mean; MCCB=Matrics Consenus Cognitive Battery; PANSS=Positive and Negative Syptom Scale; RSES=Revised Self-Efficacy Scale; SD=standard deviation; WTAR=Weschler Test of Adult Reading.
Forty-three participants (CR=22; CG=21) completed their allocated intervention. Seven CR and six CG playing participants withdrew during intervention with most either lost to follow-up or no longer interested in participating (see Figure 1). Attrition rates from baseline to end-treatment (23%) were within ranges reported in other trials (0–47.5%; Wykes et al., Reference Wykes, Huddy, Cella, McGurk and Czobor2011). Non-completers did not differ from completers in age, years of education, estimated IQ or medication dosage (all p > .05). Non-completers did, however, demonstrate significantly higher baseline self-efficacy (p=.011) and negative symptoms (p=.019) as well as trend-level differences suggestive of lower global cognitive function (p=.064) when compared with completers. No significant baseline group differences were revealed when including intervention completers only (all p > .05; see supplementary data for completer characteristics). The total number of sessions attended by CR (M=12.95; SD=3.03) and CG (M=12.86; SD=2.82) completers were equivalent (p=.914).
Measures
Primary outcome
The primary outcome was global cognition, measured using the MATRICS Consensus Cognitive Battery (MCCB; Nuechterlein et al., Reference Nuechterlein, Green, Kern, Baade, Barch, Cohen and Marder2008). The MCCB contains 10 standardized neuropsychological tasks that measure seven domains of cognition. Domain scores are used to generate a global composite (t-score: M=50; SD=10) derived from a normative sample of 300 adult controls (Kern et al., Reference Kern, Nuechterlein, Green, Baade, Fenton, Gold and Marder2008).
Secondary outcomes
Everyday living skills were measured using the Independent Living Skills Survey-Self Report (ILSS-SR; Wallace, Liberman, Tauber, & Wallace, Reference Wallace, Liberman, Tauber and Wallace2000); a 70-item self-report measure designed for use in schizophrenia. The ILSS-SR asks participants whether they completed a range of everyday tasks within the last 30 days. Items are scored as Yes (=1), No (=0) or Not Applicable with the total averaged to yield a global functioning index. Scores closer to 1 indicate higher levels of independent functioning. The ILSS-SR has been rated via expert consensus as one of the best self-report measures of independent functioning in this population (Leifker, Patterson, Heaton, & Harvey, Reference Leifker, Patterson, Heaton and Harvey2011).
Task-specific self-efficacy was assessed using a 35-item version of the Revised Self-Efficacy Scale (RSES; Cardenas et al., Reference Cardenas, Abel, Bowie, Tiznado, Depp, Patterson and Mausbach2013; McDermott, Reference McDermott1995). This self-report scale focuses on a participant’s confidence in their ability to complete everyday living and social behaviors (e.g., “attend classes” and “make friends”) (Cardenas et al., Reference Cardenas, Abel, Bowie, Tiznado, Depp, Patterson and Mausbach2013). RSES items are rated on a 5-item scale ranging from 1 (not at all confident) to 5 (extremely confident). Higher scores represent higher levels of self-efficacy.
Psychiatric symptoms were operationalized using the traditional factor structure of the Positive and Negative Syndrome Scale (PANSS; Kay, Fiszbein, & Opler, Reference Kay, Fiszbein and Opler1987). Subjective quality of life was measured using the eight-item European Health Interview Survey (EUROHIS-QOL; Schmidt, Muhlan, & Power, Reference Schmidt, Muhlan and Power2006). This scale targets physical, psychological, social, and environmental domains. Each item is rated on a 5-item scale ranging from 1 (very dissatisfied) to 5 (very satisfied). Higher scores represent better quality of life.
Procedure
Trained research staff, blinded to participant group allocation, conducted assessments at three time points: (1) baseline; (2) end-group; and (3) 3-months post-intervention. Assessors held Bachelors to Professorial level qualifications in psychology. Each outcome measure was administered at every time point. Participants were asked about alcohol or other drug use in the 24 hr preceding their assessments to screen for potential acute intoxication. On average, groups commenced within 2 weeks of participant baseline assessments (M=10.55 days; SD=7.38), while end assessments were completed within 1 week of group completion (M=5.28 days; SD=2.99).
Interventions
All participants were offered twenty 1-hr computer sessions, delivered in the community twice weekly in groups of two to five people (median=3) over 10 weeks. Participants completed their own computer exercises individually; however, sharing aspects of the learning experience was promoted as part of the group setting (Medalia & Choi, Reference Medalia and Choi2010). A doctoral-level trainee (“provisional”) psychologist (S.B.) facilitated all sessions and supported participant motivation, guided task selection, and maintained the training schedule. Participants attending at least 50% of sessions (i.e., ≥10/20) were deemed group “completers”.
Cognitive remediation
CR used COGPACK computer software (v.8.91, Marker Software®). This package contains training exercises that target areas of cognitive impairment in schizophrenia including attention, new learning, and executive functions. CR trials using COGPACK have been found to improve cognitive test performance in people with schizophrenia, including interventions with only moderate training dosages (i.e., 10–36 hr over 4–12 weeks; Cavallaro et al., Reference Cavallaro, Anselmetti, Poletti, Bechi, Ermoli, Cocchi and Smeraldi2009; Lindenmayer et al., Reference Lindenmayer, McGurk, Mueser, Khan, Wance, Hoffman and Xie2008; Sartory, Zorn, Groetzinger, & Windgassen, Reference Sartory, Zorn, Groetzinger and Windgassen2005). Training followed a semi-structured protocol with two to three cognitive domains targeted per session. Sessions generally started with exercises targeting attention and reaction speed, and ended with those tapping higher-order constructs. Task difficulty was managed automatically or manually depending on the exercise. The COGPACK program provides feedback after every exercise on relevant parameters including “percentage correct” and “completion speed”.
Drill-and-practice computer content was combined with additional instruction in internal cognitive strategies (e.g., semantic clustering, story method, imagery), which was led by the group facilitator. Strategy provision was partly informed by strategies described in the Thinking Skills for Work program (McGurk et al., Reference McGurk, Mueser and Pascaris2005, Reference McGurk, Mueser, Xie, Feldman, Shaya, Klein and Wolfe2016, Reference McGurk, Mueser, Xie, Welsh, Kaiser, Drake and McHugo2015; McGurk & Wykes, Reference McGurk and Wykes2008) and in clinical CR guidelines (Wykes & Reeder, Reference Wykes and Reeder2005); yet focused specifically on improving training task performance without extension to everyday functioning.
Strategies were particularly targeted toward new learning and executive tasks. Participants first attempted tasks independently and received performance feedback from the computer program. This was then followed by in-session discussions about relevant task-specific strategies as well as encouragement to practice these or adopt new strategies in future learning trials. Learning content was contained within each 1-hr session. Similar models of CR using COGPACK have been implemented previously (Cavallaro et al., Reference Cavallaro, Anselmetti, Poletti, Bechi, Ermoli, Cocchi and Smeraldi2009; Contreras, Lee, Tan, Castle, & Rossell, Reference Contreras, Lee, Tan, Castle and Rossell2016).
Computer games
A CG playing group was used to control for non-specific training elements including routine, social interaction and computer stimulation. Computer activities chosen were commercially available games (e.g., arcade and puzzle games) with selection based on putative similarities to CR exercises including art style and interaction levels, and their use in other trials (Fisher et al., Reference Fisher, Loewy, Carter, Lee, Ragland, Niendam and Vinogradov2015). While CG playing participants had similar opportunities to socialize with the facilitator and other group members as those in CR, and instruction and task-specific advice was available when needed, there was no provision of internal cognitive strategies in this condition.
Data Analysis
Data were analyzed using SPSS software (v.21.0). Linear mixed-effect analyses were used to examine the impact of group, time and their interaction on the principal study outcomes. For each outcome, participants with completely missing data at every time point were removed from subsequent analyses. For cognition, three participants at baseline and one participant at end-group were missing at least one MCCB domain score, resulting in an incomplete calculation of a global cognition score. Since missingness was revealed to be random (Little’s MCAR: p > .05), single imputation using expectation maximization was used to generate missing item scores estimated from all present MCCB items. This enabled computation of global cognition t-scores for these participants, which maximized the use of available observations. MCCB composites were not age and gender-corrected. This was due to our sample groups being matched for age and gender, the overall sample having a similar mean age as the U.S. normative sample, and lack of data showing whether the U.S. normative sample demographic stratification is consistent with the Australian population. This was thought to be a conservative approach to calculating t-scores.
The primary mixed-model analyses were conducted with intervention completers only. Models were run initially with fixed parameters and then reanalyzed with the inclusion of random intercepts. The repeated measurement covariance structure providing the best goodness-of-fit using Akaike Information Criterion with restricted maximum likelihood estimation was used to inform selection of the final model. Compound symmetry covariance structures provided the best fit for the global cognition and total symptom outcomes, while an unstructured configuration provided the best fit for the remaining measures. Bonferroni-adjusted pairwise comparisons were conducted to delineate significant interaction terms. Intention-to-treat (ITT) analyses using all randomized participants with at least one complete data point were used to examine the sensitivity of the completer-only models.
Reliable change indices were also calculated for individual and global MCCB cognitive domain(s) for participants providing complete baseline and end-group data (CR=22; CG=20). One CG completer was removed from this investigation, having not attempted the end-group MCCB. Calculation of reliable change indices was used to determine those who demonstrated statistically meaningful improvement over time independent of measurement reliability. This analysis approach adopted methods proposed by Jacobson and Truax (Reference Jacobson and Truax1991) with MCCB test–retest reliability estimates and standard deviations obtained from Gray et al. (Reference Gray, McMahon, Green, Seidman, Mesholam-Gately, Kern and Gold2014). A standardized cutoff of +1.65 (90% confidence interval) was used to indicate reliable improvement. Cohen’s d effect sizes (d z ) accounting for the correlation between MCCB domains at pre- and post-treatment were also estimated from this completer sample (Lakens, Reference Lakens2013).
RESULTS
Impact of Group Interventions
Results from the linear mixed-effect analyses using the completer-only sample are summarized in Table 2. Significant time (p < .001) and group × time interaction (p=.028) effects were found for the primary outcome of global cognition, indicating greater benefit for the CR group. On secondary outcomes, a significant time main-effect was evident for everyday living and social self-efficacy (p=.028) indicating both groups improved over time, but there was no significant group × time interaction. No significant effects were found for independent living skills, symptomatology, or quality of life.
Table 2 Results of linear mixed-effect analyses on primary and secondary outcomes among CR and CG completers
Note. N=43.
* p < .05.
CR=Cognitive remediation; CG=computer games; EUROHIS-QOL=European Health Interview Survey – Quality of Life; ILSS-SR=Independent Living Skills Survey – Self-Report; MCCB=Matrics Consenus Cognitive Battery; M=mean; PANSS=Positive and Negative Syptom Scale; RSES=Revised Self-Efficacy Scale; SE=standard error.
Post hoc investigation of the interaction effect for global cognition was conducted using separate linear mixed-effect analyses, with time as the fixed effect for CR and CG groups separately. Bonferroni-adjusted pairwise comparisons were calculated, with an additional adjustment as two analyses were conducted independently for the two groups (significant difference concluded if the Bonferroni-adjusted pairwise comparison p < .025).
For CR participants, global cognition was significantly better than pre-group at both end of group (p=.008) and 3-month follow-up (p < .001), whereas there was no significant difference in cognition between end-group and 3-month follow-up (p=.20). For CG participants, no significant pairwise differences were evident (all p > .30). For self-efficacy, the interaction term was not significant; however, on qualitative examination, the CR completers appeared to demonstrate a sustained improvement from end-group to 3-month follow-up, while CG participants trended toward pre-group levels after the end of group.
Re-analysis using the ITT sample was conducted to examine the sensitivity of the models when including all randomized participants. All mixed models remained significant, albeit with poorer goodness of fit, except for the general self-efficacy model, which became non-significant (p=.096).
Rates of Reliable Cognitive Improvement and Effect Size
As shown in Table 3, five CR completers (22.73%) demonstrated a statistically reliable improvement on global cognition from baseline to end-treatment in contrast with only two CG completers (9.52%). Improvement on global cognition for CR completers corresponded with a moderate effect overall (d z =0.68) whereas only a very small effect for change was found for CG completers (d z =0.06). Seventeen CR (77.27%) and nine CG (42.86%) completers improved reliably in at least one MCCB cognitive domain.
Table 3 Comparison of reliable improvements and effect sizes in MCCB domains between CR and CG completers
Note. Reliable improvement based on a reliable change index of +1.65 (i.e. 90% confidence level).
Percentage improved reported to the nearest whole number.
Reliable change index and Cohen’s d effect size estimated using the per-protocol completer sample (i.e. CR=22; CG=20).
CR=Cognitive remediation; CG=computer games; d z =Cohen’s d; MCCB=Matrics Consenus Cognitive Battery; M=mean; SD=standard deviation.
DISCUSSION
This study examined the impact of a computer-assisted CR intervention, which had been supplemented with internal cognitive strategies and delivered in a group setting, on global cognition, everyday self-efficacy and independent living skills in community-dwelling individuals with schizophrenia, when compared to an active CG playing control.
Consistent with our hypothesis, drill-and-strategy CR produced a moderate improvement in global cognition at end-treatment, which was also sustained at 3-month follow-up, with improvement greater than that achieved by an active control. This effect was observed to be relatively robust, occurring in both completer and ITT samples. In addition, three-quarters of CR completers demonstrated a statistically reliable change in at least one cognitive domain immediately following CR (c.f., ≤50% in CG), emphasizing that improvement was greater than that stemming from practice and/or non-specific effects experienced by CG completers.
Contrary to initial predictions, however, improvements in self-efficacy regarding everyday living and social abilities were comparable between CR and CG completers over time. This main effect became non-significant when all cases were estimated in the ITT sample, which may have been impacted by the complexity of non-completers who, relative to completers, had poorer cognition as well as higher levels of negative symptoms and self-efficacy. Consequently, these participants may have had less capacity for change. There was no change in self-reported independent living skills, overall symptoms, or quality of life following intervention completion.
Comparisons With Previous Research and Implications
Cognition
Overall, these findings are broadly consistent with conclusions from the most comprehensive meta-analysis, that CR can produce moderate improvements in cognitive test performance relative to control conditions at post-treatment (d=0.45), which may also be durable (d=0.43) at least over the short term (Wykes et al., Reference Wykes, Huddy, Cella, McGurk and Czobor2011). The findings also provide support for drill-and-strategy CR using COGPACK as capable of improving cognitive test performance, which in this study, was evident among completers even despite an average attendance of 13 sessions.
A decision was made to deliver an intervention that was considered feasible in a community context (Medalia & Richardson, Reference Medalia and Richardson2005; Wauchope, Terlich, & Lee, Reference Wauchope, Terlich and Lee2016); however, most previous RCTs describing cognitive benefits using this program have offered 24- to 36-hr schedules of outpatient training over 3–6 months (Cavallaro et al., Reference Cavallaro, Anselmetti, Poletti, Bechi, Ermoli, Cocchi and Smeraldi2009; Lindenmayer et al., Reference Lindenmayer, McGurk, Mueser, Khan, Wance, Hoffman and Xie2008; McGurk et al., Reference McGurk, Mueser and Pascaris2005; Vita et al., Reference Vita, De Peri, Barlati, Cacciani, Deste, Poli and Sacchetti2011). Global cognitive effect sizes may be larger following intense, restoratively based training programs (d=0.86) providing 50–100 hr over 10–20 weeks (Fisher et al., Reference Fisher, Holland, Merzenich and Vinogradov2009, Reference Fisher, Holland, Subramaniam and Vinogradov2010; Subramaniam et al., Reference Subramaniam, Luks, Fisher, Simpson, Nagarajan and Vinogradov2012). However, direct systematic comparisons of specific training approaches (i.e., drill-and-practice vs. drill-and-strategy) on training usage and outcomes are lacking.
In terms of sample characteristics, baseline cognitive ability among completers was considered fairly representative of other schizophrenia samples, with most cognitive domains falling within reported ranges of 1–2 SDs below healthy controls (Heinrichs & Zakzanis, Reference Heinrichs and Zakzanis1998). There was also consistency in global cognition levels between study completers (M=31.65; SD=11.01) and those in multi-site cognitive evaluations as well as other clinical trials in people with schizophrenia (M=26.7–35.31) (August, Kiwanuka, McMahon, & Gold, Reference August, Kiwanuka, McMahon and Gold2012; Keefe et al., Reference Keefe, Fox, Harvey, Cucchiaro, Siu and Loebel2011; Lindenmayer et al., Reference Lindenmayer, McGurk, Khan, Kaushik, Thanju, Hoffman and Herrmann2013).
PANSS total symptom ratings revealed that participants were, on average, at the upper end of “mildly ill” (i.e., 61.07) at the time of recruitment (Leucht, Reference Leucht2014); well within ranges reported in previous schizophrenia studies using COGPACK (Bosia et al., Reference Bosia, Buonocore, Bechi, Spangaro, Pigoni, Croci and Cavallaro2017; Lindenmayer et al., Reference Lindenmayer, McGurk, Khan, Kaushik, Thanju, Hoffman and Herrmann2013; Scheu et al., Reference Scheu, Aghotor, Pfueller, Moritz, Bohn, Weisbrod and Roesch-Ely2013; Vita et al., Reference Vita, De Peri, Barlati, Cacciani, Deste, Poli and Sacchetti2011). Accordingly, the CR intervention described here may be suitable for community-dwelling individuals with schizophrenia experiencing typical levels of cognitive impairment and moderate symptomatology.
Further consideration appears warranted, however, regarding factors contributing to the observed cognitive benefits for CR completers, particularly given the emphasis on task-specific instructional strategies in a relatively chronic middle-aged sample. Experiential evidence has previously highlighted the importance of instructional methods to perceived cognitive and everyday improvements in people with schizophrenia completing CR (Contreras et al., Reference Contreras, Lee, Tan, Castle and Rossell2016). In the current study, participants were encouraged to try strategies designed to improve cognitive test performance during exercises, with the expectation that techniques viewed as useful would be adopted in future learning trials and task performances become more efficient over time. In this regard, CR offered an opportunity to learn strategies that could assist participants in overcoming task-specific cognitive challenge. This model is distinct from “drill-and-practice” approaches that purport to bring about cognitive improvement via putative restorative benefits that occur during repeated practice (Vinogradov, Fisher, & de Villers-Sidani, Reference Vinogradov, Fisher and de Villers-Sidani2012).
Previous studies have suggested that shorter group-based, drill-and-practice training (e.g., 10 hr over 3–4 weeks) may be more likely to produce cognitive change in less chronic samples (Rauchensteiner et al., Reference Rauchensteiner, Kawohl, Ozgurdal, Littmann, Gudlowski, Witthaus and Juckel2011; Sartory et al., Reference Sartory, Zorn, Groetzinger and Windgassen2005). Lower intensity restorative interventions may not be sufficient for individuals with a prolonged illness course due to factors including lower potential for neuroplastic training responses, longer-term exposure to antipsychotic medications and greater disease burden (Fisher, Subramaniam, Panizzutti, & Vinogradov, Reference Fisher, Subramaniam, Panizzutti and Vinogradov2013; Rauchensteiner et al., Reference Rauchensteiner, Kawohl, Ozgurdal, Littmann, Gudlowski, Witthaus and Juckel2011; Vinogradov et al., Reference Vinogradov, Fisher, Warm, Holland, Kirshner and Pollock2009).
The fact that CR completers in this study improved their cognitive performance following a relatively short drill-and-strategy intervention raises the possibility that strategic instruction may promote near-transfer effects on cognitive tests in more chronic participants (Fiszdon et al., Reference Fiszdon, McClough, Silverstein, Bell, Jaramillo and Smith2006). As there was no comparison with a drill-and-practice training model in this study, however, we cannot be sure what aspects of the training underpinned the gains made.
Clinically, it is important to note that effective utilization of strategy-based methods may be dependent on multiple cognitive factors including functional learning and memory skills and metacognitive awareness to recognize scenarios where strategies should be applied (Farreny et al., Reference Farreny, Aguado, Corbera, Ochoa, Huerta-Ramos and Usall2016; Velikonja et al., Reference Velikonja, Tate, Ponsford, McIntyre, Janzen and Bayley2014; Wykes & Reeder, Reference Wykes and Reeder2005). Exacerbation of acute affective and/or psychotic symptoms may further complicate the learning process, as may motivational factors. Consideration of participant characteristics may be important when identifying the most appropriate cognitively enhancing intervention.
Individuals with pronounced dysexecutive symptoms, including apathy or disinhibition for example, might respond better to structured external compensatory interventions that focus more directly on real-world activities such as Cognitive Adaptation Training (Allott, Killackey, Sun, Brewer, & Velligan, Reference Allott, Killackey, Sun, Brewer and Velligan2016; Draper, Stutes, Maples, & Velligan, Reference Draper, Stutes, Maples and Velligan2009; Maples & Velligan, Reference Maples and Velligan2008). CAT aims to maximize cognitive strengths and support adaptive everyday behaviors by optimizing external strategies and environmental adaptations, while circumnavigating areas of cognitive impairment (Maples & Velligan, Reference Maples and Velligan2008). There is a significant need for research examining patient characteristics that influence treatment as well as comparisons of CR interventions with functionally grounded approaches.
The role of specific therapeutic characteristics on training-related gains may also require further consideration. It is unclear whether increasing the length and/or intensity of the current intervention would have produced greater cognitive benefits. However, Wykes et al. (Reference Wykes, Huddy, Cella, McGurk and Czobor2011) demonstrated that therapeutic parameters, such as treatment duration, may not moderate the cognitive effect sizes in CR. Additionally, although few studies have directly manipulated training exposure, a recent study comparing cognitive outcomes at 3 and 6 months of COGPACK training revealed no further cognitive improvement with the additional 3 months of training (Buonocore et al., Reference Buonocore, Bosia, Bechi, Spangaro, Cavedoni, Cocchi and Cavallaro2017). Cognitive effects of CR may, therefore, plateau once a sufficient treatment threshold has been reached (Buonocore et al., Reference Buonocore, Bosia, Bechi, Spangaro, Cavedoni, Cocchi and Cavallaro2017). Nevertheless, it cannot be determined whether participants obtained optimal training exposure for the current intervention. Identifying specific training parameters that may modify treatment outcomes, whether their impact differs according to intervention approach, and how these interact with patient characteristics (i.e., motivation), represent important areas of future investigation (McGurk et al., Reference McGurk, Mueser, Covell, Cicerone, Drake, Silverstein and Essock2013).
Independent Living Skills
There was no effect of drill-and-strategy CR on self-reported living skills despite internal strategy provision and evidence of increased cognition and everyday efficacy in this trial. This is also despite everyday living skills being arguably more proximal to the CR training exercises compared to other functional outcomes (e.g., occupational status) as well as their close association with cognitive impairment in schizophrenia (Harvey et al., Reference Harvey, Heaton, Carpenter, Green, Gold and Schoenbaum2012; Kurtz, Reference Kurtz2012). This study is, therefore, consistent with others that have failed to demonstrate generalization of gains from cognitive training to the real world in individuals with schizophrenia (Dickinson et al., Reference Dickinson, Tenhula, Morris, Brown, Peer, Spencer and Bellack2010; Fisher et al., Reference Fisher, Holland, Merzenich and Vinogradov2009; Gomar et al., Reference Gomar, Valls, Radua, Mareca, Tristany, del Olmo and McKenna2015; Murthy et al., Reference Murthy, Mahncke, Wexler, Maruff, Inamdar, Zucchetto and Alexander2012) and other patient groups (das Nair, Martin, & Lincoln, Reference das Nair, Martin and Lincoln2016; van Heugten, Ponds, & Kessels, Reference van Heugten, Ponds and Kessels2016).
Studies providing the strongest evidence for functional generalization using CR in schizophrenia have combined remediation with access to parallel intervention(s) including vocational or social cognitive training; personalized goal setting; functionally relevant strategies; and/or explicit bridging sessions to link training exercise and techniques with everyday activities (Bowie et al., Reference Bowie, McGurk, Mausbach, Patterson and Harvey2012; Lindenmayer et al., Reference Lindenmayer, McGurk, Khan, Kaushik, Thanju, Hoffman and Herrmann2013; Wykes et al., Reference Wykes, Huddy, Cella, McGurk and Czobor2011). These elements were not included in the present study. This may suggest that psychosocial disability may need to be addressed directly if the primary intervention goal is improved community functioning (Mueser, Reference Mueser2012).
An additional point of consideration, however, is that this intervention may not have been of sufficient “dose” to support improvement in everyday functioning. On average, participants only attended 13 sessions; well below commonly idealized intervention schedules in schizophrenia samples (e.g., two to three sessions/week for 3–6 months; Medalia & Richardson, Reference Medalia and Richardson2005; Mueser, Deavers, Penn, & Cassisi, Reference Mueser, Deavers, Penn and Cassisi2013). Previous studies have described functional benefits in the context of longer 24–36 hr CR intervention schedules using COGPACK (Cavallaro et al., Reference Cavallaro, Anselmetti, Poletti, Bechi, Ermoli, Cocchi and Smeraldi2009; Lindenmayer et al., Reference Lindenmayer, McGurk, Mueser, Khan, Wance, Hoffman and Xie2008; Vita et al., Reference Vita, De Peri, Barlati, Cacciani, Deste, Poli and Sacchetti2011), although the unique real world benefits of CR using this program alone are difficult to interpret given that it is often combined with functional interventions (McGurk et al., Reference McGurk, Mueser and Pascaris2005, Reference McGurk, Mueser, Xie, Welsh, Kaiser, Drake and McHugo2015).
The 3-month follow-up used in this study may also have been too short to detect changes in independent living skills. Cognitive enhancement among CR participants has been associated with greater quality of life and social functioning at 6-months post-treatment (Fisher et al., Reference Fisher, Holland, Subramaniam and Vinogradov2010; Wykes et al., Reference Wykes, Reeder, Landau, Everitt, Knapp, Patel and Romeo2007). Research exploring the impact of CR on functional outcomes when delivered as a single-modal intervention and predictors of improved community engagement is, therefore, required.
Self-efficacy
One unique study finding was that both CR and CG completers reported improvements in self-efficacy concerning their ability to successfully complete everyday living and social tasks. While few previous studies have explored whether CR can enhance everyday confidence beliefs beyond controls or whether this generalizes to broader functioning, others have investigated training-specific expectations of success (i.e., perceived competency), revealing these to be responsive to cognitive interventions and predictive of training-related improvements (Bowie et al., Reference Bowie, Grossman, Gupta, Holshausen and Best2017; Choi & Medalia, Reference Choi and Medalia2010; Saperstein & Kurtz, Reference Saperstein and Kurtz2013). Non-psychiatric models posit that self-efficacy can be enhanced through multiple pathways including the reinforcement of success during challenging tasks and perceived mastery, vicarious learning via observed peer achievements, and through verbal persuasion and titrated social support (Bandura, Reference Bandura1977). These processes may have been present in both stimulating group-based interventions in this trial.
The current study findings suggest that sustained participation in both cognitively and socially engaging programs facilitated improvements in confidence in broader functional domains, thus representing a group-related intrinsic “side-effect”. Group completers may, for example, have felt more confident in their ability to undertake activities that were part of the intervention, such as to “talk to people in a group,” “concentrate on your work,” and “get out of the house”. Nevertheless, some caution is warranted when considering the potential therapeutic benefit of these interventions in self-efficacy in the absence of a non-intervention control. Self-efficacy may have increased spontaneously in the context of treatment-as-usual.
In addition, the stability of observed enhancements in self-efficacy over the longer term remains unclear. Qualitatively, estimated marginal means suggested that, although everyday self-efficacy was higher at all time points in the CG playing condition, reported levels appeared to trend downward from end-group to follow-up in the control. This pattern was not observed in the CR group, which may suggest that the two interventions impacted on self-efficacy differently over time. Exploring the extent to which cognitively stimulating interventions can impact underlying everyday confidence beliefs, their progression over time, and their relationship with real-world functioning appears necessary.
Limitations
The following limitations must be considered when interpreting these findings. While this study aimed to gain a better understanding of community-based CR alone, the intervention did not include a direct focus on real-world generalization. This appears to have reduced potential for change in independent living skills. Without longer follow-up it is also unclear how cognitive and self-efficacy improvements progressed over time. The sample size was modest, although appeared sufficient to power the primary mixed-model analysis. In addition, while the integration of strategies into training falls within contemporary definitions of CR, the current design cannot disentangle whether computer content or strategy use, or their interaction, was responsible for the observed cognitive improvement. A single facilitator ran all of CR and CG playing groups; however, any potential bias in empathy and motivational capacity was likely systematically applied across conditions.
Furthermore, the statistical trend favoring higher self-efficacy in the control group at baseline may have masked a possible group × time interaction effect. Finally, the use of a self-report scale of living skills was potentially problematic given that completion is reliant on functional memory and introspection. Capacity-based functional measures (e.g., UCSD Performance-Based Skills Assessment), which are more strongly correlated with cognition, may be more sensitive to potential far-transfer (Elliott & Fiszdon, Reference Elliott and Fiszdon2014; Harvey, Velligan, & Bellack, Reference Harvey, Velligan and Bellack2007). In retrospect, the ILSS-SR may have been more suitable in an intervention targeting psychiatric symptoms given recently reported relationships between the ILSS-SR and emotional distress (Elliott & Fiszdon, Reference Elliott and Fiszdon2014).
Taken together, the current study provides additional evidence that CR delivered over 20 sessions in the community, even with a mean attendance of 13, may produce changes in cognitive test performance in people living with schizophrenia. Additionally, this trial suggests that stimulating group programs may produce improvements in self-efficacy regarding everyday living and social behaviors, presumably via non-specific elements including social engagement, positive learning experiences, and commitment to a stimulating community-based program. However, meaningful gains in real-world functional domains such as everyday living skills, an overarching goal of many cognitive enhancement programs, may be unlikely to occur when using this type of intervention.
Methods of achieving improved community function require further investigation. Rehabilitation approaches that focus directly on real-world tasks in an individualized manner, which may include various extents of functional compensation, are likely needed to capitalize on gains in cognitive skills and achieve a lasting reduction in psychosocial disability. There is a pressing need to examine the efficacy of such interventions to improve quality of life and optimize functional engagement for individuals with schizophrenia.
REGISTRATION DETAILS
The trial was registered retrospectively with Australian and New Zealand Clinical Trial Registry (ANZCTR, http://www.anzctr.org.au/) on 24/10/16 (trial number: ACTRN12616001476426).
ACKNOWLEDGEMENTS
The authors declare no conflicts of interest. The conduct of this research was supported by doctoral and honors funds provided by the School of Psychological Sciences at Monash University as well as an equipment grant provided by Faculty of Health, Arts and Design at Swinburne University. Shayden Bryce and Richard Lawrence are in receipt of a Research Training Program stipend funded by Monash University. Sean Carruthers is in receipt of a Research Training Program stipend funded by Swinburne University. Stuart Lee is in receipt of a National Health and Medical Research Council Early Career Fellowship. We thank all of the people that supported the conduct of this study and invested their time and effort into making it possible. We are particularly indebted to our participants and hope their experiences were beneficial in their journey toward recovery.
Supplementary Material
To view supplementary material for this article, please visit https://doi.org/10.1017/S1355617717001369
